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SUMMARY 


The  safety  and  the  effectiveness  of  patient  care  emanating  from 
AMOSIST-staf fed  Acute  Minor  Illness  Clinics  (AMICs)  was  determined  by 
evaluating  and  comparing  the  records  of  patients  treated  at  three 
AMICs  and  three  physician-staffed  general  outpatient  clinics  (GOCs) . 

The  safety  with  which  AMIC  care  was  delivered  was  assessed  by  evaluating 
the  extent  to  which  the  information  recorded  by  AMOSISTs  was  consistent 
with  the  algorithms  provided  for  their  use.  The  effectiveness  of  care 
delivered  by  AMICs  was  determined  by  comparing  the  data  of  AMIC-treated 
patients  to  that  of  GOC-treated  patients  regarding  the  rates  at  which 
these  patients  made  non-directed  returns  for  additional  care  within  14 
days  of  their  initial  visit  to  the  clinic.  The  findings  indicate  that 
the  error  rate  evidenced  by  AMOSISTs  was  unacceptably  high.  The  AMIC 
versus  GOC  comparison,  however,  yielded  no  statistically  significant 
difference  in  non-directed  patient  return  rates.  In  examining  the 
program,  it  appears  that  (a)  the  inadequate  local  auditing  procedures 
used,  (b)  the  infrequent  use  of  the  AMOSIST  Manual  documented  in  the 
first  report  of  this  series  and  (c)  the  lack  of  a data  collection  and 
recording  procedure  which  clearly  depicts  the  logic  employed  by  AMOSISTs 
in  treating  their  patients,  are  likely  to  be  the  foremost  contributors  to 
the  high  error  rates  encountered.  In  view  of  the  error  rate  observed  and 
the  non-documented  status  of  the  algorithms  employed,  it  is  asserted  that 
the  program  is  "at  risk"  from  a medico-legal  point  of  view.  Accordingly, 
it  was  recommended  that  (a)  existing  local  auditing  requirements  be  in- 
creased and  enforced,  (b)  the  existing  data  gathering  and  data  utiliza- 
tion procedures  be  replaced  by  an  integrated,  consolidated  approach, 

(c)  to  the  extent  possible,  existing  algorithms  be  replaced  with  those 
which  have  been  (or  are  to  be)  validated  during  the  course  of  the  research 
being  conducted  at  the  AMIC,  Brooke  Army  Medical  Center,  and  (d)  a more 
active,  centralized  system  of  monitoring  the  performance  of  AMOSISTs  at 
each  facility  be  implemented  within  the  Health  Services  Command. 
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1.  INTRODUCTION. 
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1.1  This  represents  the  third  in  a series  of  four  reports  on  the 
"field"  evaluation  of  the  US  Army's  AMOSIST  Program.  A more  complete 
description  of  the  background  to  the  study  is  presented  in  the  initial 
report  (Schopper,  1978a) . The  first  report  addressed  the  issues  of  the 
program's  acceptance  and  its  operational  characteristics.  The  second 
evaluated  the  extent  to  which  the  program  resulted  in  a savings  of 
physicians'  time  and  the  cost  effectiveness  of  the  program  (Schopper, 
1978b).  The  fourth  report  will  address  the  follow-up  findings  pertaining 
to  the  program's  acceptance  and  operational  characteristics. 

1.2  Briefly,  the  AMOSIST  Program  is  a relatively  recently  imple- 
mented US  Army  Program  developed  in  response  to  the  continuing  physician 
shortage.  It  is  a physician  extender  program  which  employs  physician- 
supervised  enlisted  medical  corpsmen  who  use  written  medical  algorithms 

to  provide  care  to  ambulatory  outpatients  suffering  acute  minor  illnesses. 
The  present  large  scale  study  was  initiated  in  response  to  a request  from 
the  Surgeon  General,  US  Army  following  inquiries  to  him  from  the  other 
armed  services  for  information  pertaining  to  the  AMOSIST  Program.  A 
study  protocol  was  developed  and  approved  (Schopper,  1976)  and  data 
collection  occurred  during  the  period  October  1976  through  June  1977. 

2.  OBJECTIVE. 

To  determine  the  degree  of  safety  and  effectiveness  with  which 
care  is  delivered  by  the  AMOSIST  Program. 

3.  METHODOLOGY . 

3.1  General.  Within  the  context  of  the  present  study,  the  phrase 
"safety  and  effectiveness  of  care"  is  operationally  delimited  to  two 
principal  components.  The  safety  component  involves  the  determination 
of  the  extent  to  which  the  medical  corpsmen  employed  in  the  AMOSIST 
Program,  i.e.,  AMOSISTs,  record  their  findings  and  treatment  in  accordance 
with  the  algorithms  provided  for  their  use.  The  medical  effectiveness  in- 
volves the  comparative  evaluation  of  the  non-directed  patient  return  rates 
evidenced  by  the  Acute  Minor  Illness  Clinics  (AMICs)  of  the  AMOSIST 
Program  and  those  evidenced  by  the  more  traditional,  physician-staffed, 
general  outpatient  clinics  (GOCs) . On-site  visits  of  two  weeks  duration 
were  made  to  three  GOCs  and  three  AMICs  to  collect  the  data  required  to 
address  these  issues.  The  three  AMICs  involved  had  been  among  12  re- 
ceiving on-site  visits  approximately  six  months  prior  to  the  time  the 
data  presently  addressed  were  collected.  The  three  selected  were  among 
those  whose  operations  were  judged  to  have  been  most  efficient  and  in 
greatest  accord  with  existing  Health  Services  Command  (HSC)  guidelines. 

The  three  GOCs  involved  were  selected  primarily  on  the  basis  of  the 
least  degree  of  involvement  by  physician  extenders  in  their  operations, 
and  secondarily  upon  the  reported  workload.  This  rationale  was  chosen 
in  order  to  (a)  select  GOCs  whose  operations  were  most  "traditional"  (in 
that  most  patient  care  was  physician-rendered)  and  (b)  to  maximize  the 
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amount  of  data  which  could  be  collected  during  the  course  of  each  two- 
week  data  collection  period.  Each  record  of  a patient  visit  selected 
for  inclusion  in  the  study  was  photocopied  from  the  patient's  record 
file  with  the  patient's  name  and  social  security  number  being  oblitera- 
ted from  the  photocopy  with  black,  felt-tip  marking  pens  prior  to 
placing  it  in  the  on-site  study  team's  data  collection  file. 

3.2  Algorithm  Adherence.  The  AMOSIST  Program,  as  described  in 
HSC's  Ambulatory  Patient  Care  Model  13  (APC  Model  #13,  1976),  indi- 
cates that  AMOSISTs  are  to  record  their  findings  and  treatment  for 
each  patient  they  treat  on  the  preprinted  data  collection  sheets  (DSCs) 
which  have  been  prepared  for  each  of  the  eight  general  algorithms  that 
they  employ.  For  the  present  study,  overlay  transparencies  were  pre- 
pared for  each  diagnosis  encountered  for  which  there  existed  at  least 
5 DCSs  which  met  the  following  two  criteria:  (a)  that  there  was  but 
one  diagnosis  rendered,  and  (b)  that  there  was  no  evidence  of  physician 
consultation  indicated.  The  reason  for  the  exclusion  of  those  single 
diagnosis  DCSs  which  evidenced  a physician's  involvement  is  that  from 
the  information  available  on  the  DCSs  there  is  no  way  to  determine  the 
extent  or  nature  of  the  physician's  involvement  (a  factor  which  is 
judged  to  be  a significant  deficit  of  the  present  data-recording  system). 

Since  some  (unknown)  portion  of  the  consultations  may  have  resulted  in 
a logic  overrule  by  the  physician,  there  was  no  alternative  but  to  ex- 
clude all  DCSs  which  evidenced  a physician  consultation  from  the  present 
analysis.  For  the  present  analyses,  the  sample  of  DCSs  screened  for 
these  characteristics  (AMOSIST-only , single  diagnosis)  included  not 
only  those  DCSs  obtained  for  the  purpose  of  the  retrospective  audit,  but 
also  those  DCSs  from  the  medical  records  of  patients  treated  in  the  AMICs 
during  the  period  the  study  team  was  on  site.  The  overlays  were  applied  to 
all  DCSs  meeting  the  requirements  and  the  errors  found  in  each  were  recorded 
and  analyzed.  It  is  noted  that  the  utilization  of  the  "single  diagnosis" 
criterion  results  in  a "simplest  case"  analysis. 

3.3  Return  Rate  Comparison. 

3.3.1  The  comparison  of  return  rates  between  AMICs  and  GOCs  was  accom- 
plished by  a retrospective  audit  of  medical  records  from  two  of  the  three 
AMICs  and  all  three  GOCs.  The  exclusion  of  the  records  from  the  third  AMIC 
was  due  to  the  finding  that  during  the  interim  between  the  initial  Phase  I 
site  visits  and  the  Phase  II  visits,  the  operation  of  the  AMIC  and  medical 
practice  of  physicians  and  physician  assistants  (PAs)  general  medicine  clinic 
became  intermixed.  As  a result,  the  sample  of  records  returned  for  the  purpose 
of  retrospective  audit  contained  a substantial  portion  of  records  from  non-AMIC 
care  providers  and  DCSs  evidencing  consultations  by  PAs  and  physicians  whose 
relationship  to  the  AMIC  was  unknown. 

3.3.2  The  records  included  in  the  sample  from  each  included  clinic 
were  randomly  selected  from  those  of  the  patients  who  had  received  treat- 
ment in  the  clinic  during  a three-week  period  beginning  six  weeks  prior 
to  the  date  of  the  arrival  of  the  on-site  study  .cam.  Upon  arrival  of 
the  team,  the  records  of  these  patients  were  first  examined  to  determine 
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which  entry  related  to  the  first  visit  to  have  occurred  within  the 
three-week  period.  The  records  were  then  examined  further  to  determine 
if  there  had  to  be  a succeeding,  non-directed  visit  to  the  hospital 
(i.e.,  a visit  not  requested  by  the  original  care  provider)  within  two 
calendar  weeks  of  the  aforecited  visit.  If  there  was,  the  entry  appro- 
priate to  this  visit  was  also  copied.  All  records  which  had  been  copied 
were  then  returned  to  the  study  agency  for  further  examination  to  deter- 
mine, for  those  patients  who  had  returned  within  the  designated  two-week 
period,  whether  or  not  the  succeeding  visit  was  an  illness  or  medical 
condition  which  could  be  considered  to  be  related  to  the  initial  visit. 
All  return  visits  which  were  related  to  the  initial  visit  were  counted 
as  a "criterion"  return  visit.  The  data  obtained  from  these  clinics 
were  then  evaluated  to  determine  if  there  existed  a statistically  signi- 
ficant difference  in  the  return  rates  evidenced  between  them. 

4.  RESULTS  AND  DISCUSSION. 

4.1  General.  The  findings  which  pertain  to  the  overlay  evaluation 
are  presented  first  (paragraph  4.2).  The  findings  which  relate  patient- 
return  rates  follow  (paragraph  4.3). 

4.2  Algorithm  Adherence. 

4.2.1  General.  In  the  process  of  selecting  the  DCSs  for  evaluation 
and  subsequently  evaluating  those  which  were  selected,  several  types  of 
information  were  developed.  Presented  below  are  the  findings  which  per- 
tain to  the  following  information:  presence  of  vital  signs  on  DCSs, 
asterisked  physician  consultation  requirements,  logic-related  physician 
consultation  requirements,  utilization  of  DCSs,  and  non-required  DCS 
entries. 


4.2.2  Vital  Signs  Information. 

4.2.2. 1 One  set  of  information  required  on  all  the  DCSs  is  that 
pertaining  to  the  recording  of  the  patient’s  vital  signs,  i.e.,  temper- 
ature, pulse  rate,  respiration  rate  and  blood  pressure.  Table  1 depicts 
the  extent  to  which  AMOSISTs  did  record  such  information. 

4. 2. 2. 2 It  is  clear  that,  with  the  exception  of  temperature  re- 
cordings, the  AMOSISTs  did  not  do  an  acceptable  job  of  recording  vital 
signs  information  on  their  DCSs.  On  none  of  the  282  single-diagnosis 
DCSs  evaluated  was  the  pulse  rate  recorded,  and  on  but  two  (0.7%)  of  them 
was  the  patient's  respiration  rate  noted.  Blood  pressure  recordings 
were  made,  on  the  average,  for  only  one  patient  in  every  eight  (12.6%). 
Even  the  most  frequently  recorded  vital  sign,  temperature,  was  absent 
from  the  patient's  DCS  in  nearly  forty  percent  of  the  cases  (39.1% 
absent) . 


4. 2. 2. 3  Several  observations  are  given  regarding  these  findings  and 
the  pattern  observed  therein.  First,  as  regards  the  apparently  poor 
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Table  1 

PRESENCE  OF  VITAL  SIGNS  INFORMATION  ON 
AMOSISTS  DATA  COLLECTION  SHEETS 


DIAGNOSIS  (#  of  Casee 


Conjunct lvltla,  bacterial  ( 8) 

Acuta  Otitla  Madia  (16) 

Acute  Serous  Otitla  Media  (14) 

Acute  Otitis  Externa  ( 8) 

Acute  Sinusitis  (10) 

Allergic  Rhinitis  (16) 

Rhinitis,  Non-specific  ( 9) 

Acute  Pharyngitis,  possibly  (17) 

streptococcal 

Acute  Pharyngitis,  Non-specific  (52) 
Bronchitis  (20) 

Headache  ( 7) 

Gastroenteritis  (17) 

Vaginitis,  Non-specific  (16) 

Cystitis  (40) 

Acne  (13) 

Contact  Dermatitis , Plant  (19) 

TOTAL  (282) 


* Data  are  depicted  in  terms  of  the  percent  of  the  total  number  of  cases 
evaluated  for  each  diagnosis. 


’ VITAL  SIGN  * 

TEMPERATURE 

PULSE 

RESPIRATION 

BLOOD  PRESSURE 

37.5 

0 

0 

0 

62.5 

0 

0 

18.8 

35.7 

0 

0 

21.4 

85.7 

0 

0 

0 

90.0 

0 

0 

20.0 

69.2 

0 

0 

23.1 

66.7 

0 

11.1 

22.2 

62.5 

0 

0 

• 

5.9 

60.0 

0 

0 

17.3 

68.4 

0 

0 

15.0 

28.6 

0 

0 

28.6 

73.3 

0 

0 

0 

43.8 

0 

0 

0 

45.0 

0 

0 

5.0 

84.6 

0 

0 

0 

83.3 

0 

5.3 

26.3 

60.9 

0 

0.7 

12.6 

I 
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overall  performance  of  AMOSISTs  on  this  task,  it  might  be  offered,  in 
their  defense,  that  such  information  reflects  a redundant  requirement 
within  the  patient's  medical  records  for  that  particular  visit.  It  was, 
in  fact,  frequently  the  case  that  at  least  some  of  the  patient's  vital 
signs  (temperature,  blood  pressure,  and,  often,  pulse  rate)  were  deter- 
mined and  recorded  on  either  a separate  SF  600  or  Triage  DCS  prior  to  the 
time  the  patient  saw  the  AMOSIST  himself.  In  such  instances,  it  is 
clearly  duplicative  to  require  the  AMOSIST  to  transfer  this  information 
to  the  DCS  that  he  will  employ  in  treating  the  patient.  Hence,  the 
omission  of  vital  signs  information  in  such  instances  would  not  appear 
to  constitute  a cause  for  serious  concern  — as  long  as  such  pre- 
treatment materials  remain  a part  of  the  patient's  medical  records. 

4. 2. 2. 4 The  caveat  just  cited  is  not  employed  as  rhetoric.  The 
status  of  the  Triage  DCS  as  a permanent  part  of  the  patient's  record 
has  been  questionable  in  the  past  (as  has  been  the  status  of  the  treat- 
ment DCS) . It  is  only  somewhat  recently  that  this  issue  has  been 
acknowledged  and  addressed.  Whereas  the  guidance  provided  in  HSC's 

APC  Model  // 13  (Aug  76)  indicates  that  both  the  triage  and  treatment  j 

DSCs  are  to  be  placed  in  the  patient's  records,  such  guidance  conflicted 
with  the  more  binding  requirements  provided  in  AR  40-400  and  AR  40-403. 

Collectively,  these  regulations  state  that  only  Department  of  the  Army  “ 

(DA)  forms  may  be  included  in  patient  records.  Since  both  the  triage 
data  collection  sheets  and  the  treatment  data  collection  sheets  are 
locally  reproduced  forms,  they  could  not  be  included  in  the  records. 

However,  a relatively  recent  interim  change  (March  1977)  to  these 
regulations  now  permits  them  to  be  included.*  Nevertheless,  based 
upon  observations  made  during  the  course  of  the  present  study,  it  is 
clear  that  such  guidance  is  not  being  uniformly  adhered  to.  One  princi- 
pal objection  which  has  been  voiced  to  the  inclusion  of  the  Triage  DCS 
in  the  medical  records  is  the  bulk  it  adds.  In  lieu  of  this  form  (a 
full  size,  one  sided  page),  one  site  made  entries  on  the  SF  600  to  re- 
cord triage  information  and  whatever  vital  signs  which  were  routinely 
obtained.  The  difficulty  observed  with  this  alternative  procedure  is 
that  this  separate  SF  600  entry  was  often  separated  from  the  treatment 
DCS  employed  for  that  visit  by  one  or  more  pages.  (In  other  words, 

SF  600  entries  were  made  sequentially  until  both  sides  of  the  form  had 
been  filled,  whereas  a new  treatment  DCS  was  initiated  for  each  visit. 

Hence,  it  was  possible  for  several  treatment  DCSs  to  appear  between  the 
last  employed  DCS  and  the  corresponding  triage/vital  signs  note  appro- 
priate to  same.)  While  it  is  acknowledged  that  such  a practice  does 
not  constitute  a breach  of  requirements,  it  does  impose  a small  amount 
of  inconvenience  to  the  AMOSIST  who  must  now  leaf  through  the  inter- 
vening pages  to  obtain  such  information  and,  hopefully,  to  transfer  it 


*The  interim  change  to  AR  40-400  has  been  incorporated  into  the  newly 
published  version  of  this  regulation  effective  1 October  1978. 
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to  his  own  treatment  DCS.  The  danger  in  such  a practice  (and  the  danger 
inherent  in  not  routinely  demanding  that  AMOSISTs  make  such  transfers) 
is  that  during  hours  of  peak  workload,  an  AMOSIST  may  neglect  to  "look 
up"  vital  signs  information  to  determine  whether  or  not  any  one  of  them 
were  abnormal  (as  defined  by  the  ranges  indicated  in  their  AMOSIST  • 

manuals) . 

4. 2. 2. 5 Of  some  additional  interest,  perhaps,  is  the  pattern  of 
omissions  observed  in  the  vital  signs  data.  For  all  practical  purposes, 
both  pulse  rate  and  respiration  rate  were  ignored  and  blood  pressure 
was  recorded  infrequently.  Only  the  patients'  temperatures  were  recorded 
with  some  degree  of  regularity  (60.9%).  While  the  reasons  for  these 
differences  cannot  be  determined  from  the  present  study,  there  do  appear 
to  be  some  plausible  speculations.  First,  it  is  noted  that,  aside  from 
the  "laundry-list"  of  symptoms  and  conditions  that  routinely  appears  at 
the  beginning  of  each  major  algorithm  within  the  AMOSIST  Manual,  it  is 
only  the  patient's  temperature  which  is  occasionally  alluded  to  in  sub- 
sequent logic  blocks  for  a few  of  the  diagnoses.  Hence,  it  is  likely 
that  the  AMOSIST,  by  virtue  of  his  continuing  experience  with  the 
algorithms,  perceives  that  information  relating  to  pulse  rate,  respira- 
tion rate,  and  blood  pressure  will  have  much  less  impact  upon  the 
eventual  diagnosis  and  treatment  than  will  information  regarding  the 
patient's  temperature.  Therefore,  the  AMOSIST  may  pay  more  attention 
to  (and  record)  this  vital  sign  more  so  than  the  others.  Another  pos- 
sibility, one  which  may  act  separately  and  in  addition  to  that  just 
cited,  is  that  concerning  the  base  rate  with  which  deviations  occur 
among  the  patient  population.  Whereas  there  is  substantial  latitude 
among  healthy  individuals  regarding  the  range  of  values  which  may  be 
considered  "normal"  for  one's  pulse  rate,  respiration  rate,  and  blood 
pressure,  the  body's  homeostatic  mechanisms  are  such  that  (for  the  most 
part)  regardless  of  an  individual's  sex,  age,  or  degree  of  physical  fit- 
ness, one's  body  temperature  remains  within  a small  stable  range  unless 
he  is  ill.  While  minor  illness  may  have  an  impact  upon  an  individual’s 
pulse  rate,  respiration  rate  or  blood  pressure  they  are  rarely  driven 
beyond  the  rather  wide  range  of  values  which  is  considered  "normal." 

Hence,  from  this  separate  vantage  point;  i.e.,  independent  of  the  fre- 
quency of  occurrence  within  the  algorithms,  it  is  likely  that  AMOSISTs 
would  more  frequently  derive  information  which  would  be  of  consequence 
to  their  work  from  a patient's  temperature  than  they  would  from  one  of 
the  remaining  vital  signs,  and,  as  a consequence,  they  are  more  likely 
to  attend  to  same. 

4.2.3  Algorithmic  Errors 

4. 2. 3.1  General..  The  analysis  of  principal  concern  in  the  present  evalua- 
tion is  that  performed  upon  the  DCSs  themselves  to  determine  to  what 
extent  they  have  been  completed  in  accordance  with  the  guidance  provided 
in  the  algorithms.  As  indicated  within  the  approved  study  protocol 
(Schopper,  1975),  this  evaluation  was  confined  to  those  DCSs  which  indi- 
cated no  physician  involvement  and  had  but  one  diagnosis  indicated. 
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These  limitations,  which  clearly  excluded  the  more  difficult,  multi- 
diagnosis cases  from  the  analyses,  derived  from  (a)  the  fact  that  once 
a physician  countersigns  a DCS,  there  is  no  way  to  determine  whether  a 
departure  from  the  algorithm's  logic  was  or  was  not  directed  by  the  con- 
sulting physician,  and  (b)  the  necessity  of  having  to  develop  an  indivi- 
dual DCS  overlay  transparency  for  each  possible  diagnosis.  Among  all  of 
the  1909  AMOSIST-completed  DCSs  from  two  AMICs  (and  those  meeting  the 
cited  criteria  from  the  DCSs  completed  during  the  on-site  visit  to  the 
AMIC  for  which  the  retrospective  audit  records  were  unusable)  only  328 
met  these  criteria.  Among  these,  only  282  occurred  with  a frequency  of 
five  or  more  and  were  subjected  to  an  overlay  evaluation. 

4. 2. 3. 2 Distribution  Among  Diagnostic  Categories.  As  shown  in 
Table  2,  these  282  cases  were  distribued  unequally  among  16  diagnosis. 
Only  two  diagnoses,  non-specific  Acute  Pharyngitis,  and  Cystitis, 
occurred  with  a frequency  greater  than  30  (52  and  38,  respectively). 
Clearly,  the  majority  of  them,  179  (63.5%),  were  URI  illnesses. 

4. 2. 3. 3 Principal  Categories  of  Errors.  Two  general  categories  of 
errors.  Type  A and  Type  B,  were  developed  to  facilitate  the  presentation 
of  the  error  related  data. 

4.2. 3. 3.1  Type  A Errors.  Type  A errors  are  those  which  pertain 

to  the  specially  marked  items  (identified  with  an  asterisk)  appearing  on 
each  DCS.  In  general,  the  Type  A,  asterisked-entry  errors  relate  to 
symptoms  or  conditions  which  are  rather  independent  of  the  logic  speci- 
fic to  any  particular  diagnosis  addressed  on  a given  DCS.  The  purpose 
of  most  of  these  items  is  to  either  (a)  identify  the  conditions  arising 
from  past  medical  history  or  treatment  which  might  interact  with  the 
symptoms/conditions  which  are  now  of  most  immediate  concern  to  the 
patient  in  such  a way  as  to  escalate  the  danger  of  possible  complica- 
tions or  demand  medical  sophistication  in  the  development  of  a treat- 
ment plan  that  was  beyond  the  capability  of  the  AMOSIST,  or  (b)  identify 
symptoms/conditions  of  such  severity  that  they  were  beyond  the  scope  of 
illnesses  addressable  by  an  AMOSIST.  As  such,  the  significance  for  an 
AMOSIST  of  the  asterisk  next  to  a DCS  data  entry  point  was  to  direct 
him  to  seek  the  consultation  of  a physician  prior  to  formulatine  a diag- 
nosis and  treatment  plan. 

4. 2. 3. 3. 2 Type  B Errors. 

4. 2. 3. 3. 2.1  In  contrast  to  the  Type  A errors,  the  Type  B error  is 
tied  directly  to  the  logic  of  the  algorithm  itself.  Such  an  error  indi- 
cates that  an  AMOSIST  has  made  an  incorrect  DCS  entry,  one  which  is  in- 
consistent with  the  decision-tree  logic  which  is  demanded  in  order  to 
arrive  at  the  diagnosis  which  has  been  rendered. 

4. 2. 3. 3. 2. 2 Also  counted  as  an  error  (in  consonance  with  guidance 
from  the  AMOSIST  Section,  Medicine  and  Surgery  Division,  Academy  of 
Health  Sciences,  AHS)  was  the  absence  of  any  response  (either  positive 
of  negative)  for  those  items  specifically  addressed  in  the  algorithm 
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Table  2 

DISTRIBUTION  OF  SINGLE-DIAGNOSIS,  AMOS I ST-TREATED 
PATIENTS  ACCORDING  TO  DIAGNOSIS 


DIAGNOSIS 

Number 
of  Cases 

Number 
of  Cases 

Conjunctivitis,  bacterial 

8 

Acute  Pharyngitis,  Non-specific 

52 

Acute  Otitis  Media 

16 

Bronchitis 

20 

Acute  Serous  Otitis  Media 

14 

Headache 

7 

Acute  Otitis  Externa 

8 

Gastroenteritis 

17 

Acute  Sinusitis 

10 

Vaginitis,  Non-specific 

16 

Allergic  Rhinitis 

16 

Cystitis 

38 

Rhinitis,  Non-specific 

9 

Acne 

13 

Acute  Pharyngitis,  possible 
streptococcal 

17 

Contact  Dermatitis,  Plant 

19 

TOTAL 


(282) 


which  required  a negative  response  to  arrive  at  the  diagnosis  cited. 

Not  counted  as  an  error,  however,  was  the  absence  of  an  entry  to  a 
DCS  entry  position  if  the  corresponding  symptom  or  condition  was  not 
specifically  addressed  within  the  algorithm.  In  the  strictest  sense, 
this  is  contrary  to  the  resident  AHS  instruction  initially  provided 
each  AMOSIST  since  each  AMOSIST  is  taught  to  make  a written  response 
(either  positive  or  negative)  to  all  DCS  entry  spaces.  However,  it 
was  observed  with  some  frequency  that  AMOSISTs  would  make  entries 
only  if  they  were  positive  and  leave  blank  all  negative  findings.  To 
have  counted  as  errors  all  such  absences  of  negative  reports  would 
have  substantially  inflated  the  number  of  errors  encountered;  therefore, 
as  indicated  above,  the  absence  of  a negative  finding  was  counted  as  an 
error  only  if  the  algorithm  specifically  addressed  a given  symptom  or 
condition. 


4. 2. 3. 3. 2. 3 Prior  to  ending  the  present  introductory  discussion 
of  Type  B errors,  two  additional  comments  are  required.  The  first  re- 
lates to  the  entry  in  the  upper  left-hand  corner  of  all  DCSs  which  per- 
tains to  the  instructions  provided  to  patients  regarding  the  possible 
need  for  them  to  return  to  the  clinic  to  obtain  diagnostic  test  results 
of  follow-up  evaluations  for  care.  Entries  at  this  location  were  ig- 
nored during  the  course  of  the  overlay  evaluation  unless  the.  algorithm 
provided  specific  instructions  regarding  the  patient's  return.  In  these 
instances,  however,  the  block  was  monitored  not  only  to  determine  if  it 
was  appropriately  checked,  but  also  to  see  if  the  time  period  indicated 
was  in  accord  with  the  instruction  contained  in  the  algorithm. 

4. 2. 3. 3. 2. 4 As  a cautionary  note  to  the  reader,  the  labelling  of 
Type  A and  Type  B errors  as  such  is  not  meant  to  imply  the  existence  of 

a hierarchy  of  differential  importance  between  them.  While  Type  A errors 
(which  pertain  to  asterisked  entries  and,  therefore,  directly  relate  to 
the  absence  of  a needed  physician  involvement)  might  appear  to  assume  a 
higher,  more  serious  status,  it  is  pointed  out  that  Type  B,  logic-related 
errors  reflect  information  or  actions  which  were  inconsistent  with  the 
algorithm  and  should  have  resulted,  as  a minimum,  in  a physician's  con- 
sultation and  might  have  resulted  in  either  (a)  an  additional  or  different 
diagnosis  than  the  one  actually  rendered  or  (b)  a different  treatment 
plan  than  that  actually  employed.  The  assertion  of  the  need  for  a 
physician's  involvement  derives  from  the  fact  that  treatment  which  is 
rendered  solely  by  an  AMOSIST  must  be  limited  to  those  specific  condi- 
tions which  are  entirely  compatible  with  the  logic  of  the  algorithm 
which  yields  those  diagnoses.  Since  Type  B errors  indicate  that  the 
symptoms  and/or  the  treatment  were  not  in  entire  agreement  with  the 
algorithm  appropriate  to  the  diagnosis  checked,  the  AMOSIST  should  have 
sought  the  counsel  of  a physician.  Clearly,  both  types  of  errors,  Types 
A and  B,  reflect  errors  of  omission — the  absence  of  a needed  consulta- 
tion with  a physician. 

4.2. 3.4  Frequency  of  Errors. 
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4.2. 3.4.1  Frequency  distribution  of  Type  A errors.  The  frequency 
distribution  for  all  Type  A,  asterisked-entry  errors  is  shown  in  Table  3. 
Clearly,  for  only  about  one  in  every  four  DCSs  is  it  the  case  that  there 
are  no  Type  A errors.  Conversely,  nearly  37  percent  (36.9%)  of  the  DCSs 
had  two  or  more  Type  A errors  present.  Overall,  73  percent  of  the  DCSs 
had  entries  associated  with  these  clearly  marked  data  entry  points  that 
should  have  resulted  in  either  an  AMOSIST's  consultation  with  a super- 
vising physician  or  the  complete  transfer  of  responsibility  for  that 
patient  to  a physician. 

4. 2. 3. 4. 2 Frequency  distribution  of  Type  R errors.  Table  4 de- 
picts the  frequency  distribution  of  Type  B logic-related  errors.  The 
figures  portrayed  therein  show  that  less  than  ten  percent  of  the  DCSs 
evaluated  were  totally  free  of  this  type  of  error.  Only  13  percent  of 
the  sample  possessed  but  a single  error  while  slightly  more  than  one- 
half  (51.5%)  evidenced  three  or  more  such  errors. 

4. 2. 3. 4. 3 Conjoint  Consideration  of  Type  A and  Type  B Errors. 

The  figures  cited  in  the  two  previous  paragraphs  and  tables  are  the  re- 
sults of  separate  analyses  performed  upon  the  282  cases  comprising  the 
sample  evaluated.  When  these  errors  are  considered  in  conjoint  fashion, 
the  overall  result  is  that  nearly  all  DCSs,  97.1  percent  of  them,  evi- 
denced at  least  one  Type  A or  Type  B error,  i.e.,  only  eight  out  of 
the  sample  of  282  cases  were  entirely  free  from  both  of  these  types  of 
errors. 

4. 2. 3. 5 Type  A and  type  B Error  Subcategories. 

4. 2. 3. 5.1  Subcategories  Delineations.  It  is  possible  to  further 
categorize  the  error-related  data  into  two  additional  major  sub- 
categories: symptom-related  errors  and  treatment-related  errors.  Both 
Type  A,  asterisked-entry  errors  and  Type  B,  logic-related  errors  are 
possible  within  each.  Each  of  these  principal  subcategories  can  be 
further  categorized  to  reflect  the  same  categories  utilized  by  the 
AMOSIST  in  performing  his  examination,  developing  a treatment  program, 
and  deciding  the  patients  disposition.  These  categories,  symptom- 
related  (subjective  and  objective)  and  treatment-plan-related 
(prescriptions,  diagnostic  tests  requested,  and  disposition)  are  con- 
sistent with  those  which  appear  on  the  DCS  itself.  Figure  1.  Among 
the  treatment-plan-related  subcategories,  however,  Type  A errors  are 
possible  only  with  the  prescription-related  and  disposition-related 
areas;  i.e.,  there  are  no  asterisked  entries  on  the  DCSs  which  are  re- 
lated to  requests  for  disgnostic  tests  or  consultations.  The  findings 
pertaining  to  each  subcategory  are  shown  in  Table  5. 

4. 2. 3. 5. 2 Symptom-Related  Errors.  Among  the  symptom-related 
errors,  Type  A,  asterisked-entry  errors  occurred  more  than  three  times 
as  often  among  the  subjectively  reported  data  than  they  did  among  the 
objectively  observed  data.  The  Type  B,  logic-related  errors  occurred 
with  nearly  equal  frequency  in  both  the  subjective  and  objective  sub- 
categories of  the  symptom-related  error  data.  As  shown  in  the  table, 
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FREQUENCY  DISTRIBUTION  OF  TYPE  A ERRORS 


NUMBER  OF 
ERRORS  PER  DCS 

NUMBER  OF 
DCSs 

RELATIVE 
FREQUENCY  (%) 

CUMMULATIVE 
FREQUENCY  (%) 

1 

102 

36.2 

36.2 

2 

67 

23.8 

59.9 

3 

27 

9.6 

69.5 

4 

8 

2.8 

72.3 

5 

2 

.7 

73.0 

27.0  100.0 


0 (no  errors)  76 


Table  4 


FREQUENCY  DISTRIBUTION  OF  TYPE  B ERRORS 


NUMBER  OF 

NUMBER  OF 

RELATIVE 

CUMMULATIVE 

ERRORS  PER  DCS 

DCSs 

FREQUENCY  (%) 

FREQUENCY  (%) 

1 

37 

13.1 

13.1 

2 

69 

24.5 

37.6 

3 

64 

22.7 

60.3 

4 

37 

13.1 

73.4 

5 

26 

9.2 

82.6 

6 

12 

4.3 

86.9 

7 

3 

1.1 

88.0 

8 

2 

.7 

88.7 

9 

- 

- 

88.7 

10 

2 

.7 

89.4 

11 

2 

.7 

90.1 

0 (no  errors) 

28 

9.9 

100.0 
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Figure  1 
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CHIEF  COHPLAIHT(S): 

SCTJECTIVl n*x* 


Chief  complaint  correct  At  Triage 

*  Return  vlclC  for  unchanged  problem 

*  Possibly  pregnant 

*  Taking  MD  ordered  med: 

*  Contraindicated  med: 

*  Shaking  chills 

*  Pev  for  mor  thn  5d  THROAT  k NEC* 

*  Rash  for  lea  thn  5d  Poa  TC  in  last  10d 

EAR  Poa  TC  in  faa  now 

Tinnitus* severe  Paat  rheum  fever 

Vertigo* severe  Sore  throat  for: 

Earache  for:  Hoarseness  for: 

Decreased  hearing  CHEST 


RESPIRATORY 

— OBJECTIVE 

DAY K> YR TIKE  IN 

TIKE  SEEK TIKE  OUT 

TEMP ,_Tf  PULSE b/m 

RESP b/m  BP / / — 

• Abno rma  1 VS  * Appears  seversly  ill 

LA*  110SE  k SINUS 

Haatold  tsnd* sev  Nasal  drain watery 

Ear  dralnaga  purul* act  bleed 

• bloody  Kasai  sue  avol  A rad 

white-yellow  • Pacial  availing 

TM  not  visible  Sinus  tsndcrnasa 

can  fil  with  wax  THROAT  4 NEC* 

TH  loss  of  landmark  Red  pharynx 

TM  redneee  Tonsillar  exudata 

TM  air  fluid  level  • Perl  tone! liar  swell 


Ear  itching  * Shortneaa  of  b 

NOSE  4 SINUS  Cough  for: 

Nasal  congestion  not  lmprovlo 

Nasal  drain purul  • prod  pur/bld 

recurrent /season  Chest  pain ml 

watery 

Itching  e yes/noee  OTHER 

Tearing  Headache  for: 

# Sin  aray  in  last  Imp*  Other: 

Runny  nose  fo' : 


1 Shortness  of  breath  TM  redness 

Cough  for:  TM  sir  fluid 

not  improving  TM  bubbles 

• prod  pur/bld  sput  • TM  retracted 

Chest  pain  mild  * TM  scar 


TM  bubbles  Lymph  nodes* poet 

•  TM  retracted  tender  anterior 

•  TM  scar  CHEST 

•  TM  perforated  #• Unequal  brth  sounds 

Canal  tenderness  #* Rales  rhoo  or  wheel 

Canal  swollen red  # Chase  point  tender 

local*  pointing  OTHER 

outer  half  #* Decieaa  neck  flax 

Ear  f lseuraa/acales* Other: 


Labyrinthitis  Ac  Otitis  Ext 

Ac  Otitis  Media  Chr  Otlt  Ext 

Ac'  Set  Otlt  Med  Ac  Sinusitis 

Puruncls  Allerg  Rhinitis  __ 

TREATMENT 

*  Temporary  profile  for:  Typ-: 

•  Quarters  for: 

Return  to  clinic  on: 

Pen  V 250  mg  #40  1 qid  alOd 

Erythro  230mg  #40  1 qid  xlOd 

Cortlaporin  Otic  lOcC  4gtt  qid  xlOd 

Hydrocort  Cm  IT  30  CM  Apply  bid 

Aspirin  325  mg  #30  2 q6h  pro 

Acetaminophen  325  mg  #30  2 q6h  pro 

Cepacol  Lot  #24  1 q4h  pro 

Robltueslo  DM  4oa  1 tap  q4h  pro 

Dlmanhydrl  50  mg  #12  1 q6h  pro 

Chlorpheo  mal  4 mg  #30  1 q6h  pro 


-ASSESSMENT 

Rhinitis,  NOS  Laryngitis  * Other: 

Strep  Sore  Thr  Costochondritis 

Ac  Pbsr.Pos  Str  Bronchitis 

Ac  Phar.NOS  Headache 

Act  1 fed  #30  1 tid 

Dlmctapp  #20  1 bid 

Neoeynephrloe  0.25Z  IScc  2gtt  qid  xlOd 

•  Other  than  AMOSIST  Manual: 

CONSULTATIONS  4 REFERRALS 

Throat  culture  Ear  culture 

Ear  canal  lrrlgat  Honoapot 

Sinus  x-raya  Nose  culture 

Cheat  X-ray 
DISPOSITION 

Duty  or  home 

•  Hospital  ward: 

• Other: 


SEEN  BY  aposlat m.d. 

O-no  /+-yes/R-+  rt  slde/L— » It  alde/RL-*  both  sides/#-do 


PULL  NAME: 

SSN / / / FIRST  VISIT  yea  no 

BIRTH  YI SEE  a f STATUS  spona  dep 

SPONS  act  ret  res  oth  SERVICE  ar  af  mar  new  oth 

RAN*:  UNIT: 

TELEPHONE  home / work  / 


Table 

5 

CATEGORIZATION  OF  TYPE 

A AND  TYPE 

B ERRORS 

BY  TYPE  OF 

DCS  ENTRY 

CATEGORY  OF 

DCS  ENTRY 

TYPE  I 

ERRORS 

TYPE 

II  ERRORS 

tom  - Related  Errors 


Subjective  Symptom 
Objective  Symptom 

Subtotal 

Treatment  - Related  Errors 


Prescription 

45.8% 

35.9% 

Diagnostic  Tests/ 
Consultations 

N/A 

7.8% 

Disposition 

0.0% 

2.3% 

Subtotal 

45.8  % 

46.0% 

TOTAL 

100.0% 

100.0% 

nearly  54  percent  of  all  Type  A and  Type  B errors  occurred  within  the 
symptom-related  portion  of  the  DCS. 

4. 2. 3. 5. 3 Treatment-Related  Errors.  In  complementary  fashion,  the 
proportions  of  Type  A and  Type  B errors  detected  in  the  treatment-related 
portion  of  the  DCS  were  also  very  nearly  equal  to  one  another.  As  regards, 
the  treatment-related  Type  A,  (asterisked-entry)  errors,  all  were 
prescription-related.  For  the  most  part,  however,  these  errors  reflected 
the  absence  of  a required  physician  consultation  to  approve  the  pre- 
scription of  one  or  more  medications  not  appearing  on  the  DCS.  In  most 
of  these  instances,  however,  the  prescribed  medications  were  viewed  as 
being  similar  in  their  effects  to  the  logic-directed  medication.  A simi- 
lar disclaimer  is  appropriate  to  the  Type  B,  logic-related  prescription 
findings.  An  examination  of  these  latter  errors  revealed  that  most  of 
them  were  due  either  to  (a)  the  lack  of  a prescription  for  symptomatic 
relief  medication  (such  as  aspirin) , or  (b)  the  use  of  the  logic  desig- 
nated "alternate"  drug  in  the  absence  of  any  contraindication  for  the 
'Brug-of-choice"  (such  as  prescribing,  in  the  absence  of  physician 
direction,  acetaminophen  instead  of  aspirin  or,  in  the  more  serious 
instances,  ampicillin  instead  of  sulf isoxazole) . 

4. 2. 3. 6 Findings  Pertaining  to  Specific  Algorithms  and  Diagnoses. 
Further  particulars  regarding  Type  A errors  for  the  URI  algorithm  are 
found  in  Appendix  A.  Additional  information  and  comments  pertaining 

to  Type  B,  logic-related  errors  for  individual  URI  diagnoses  are  pro- 
vided in  Appendix  B.  The  description  focuses  upon  the  most  frequently 
occurring  diagnosis,  non-specific  acute  pharyngitis.  An  extended  dis- 
cussion of  Type  B,  logic-related  errors  for  the  second  most  frequently 
occurring  diagnosis  in  the  entire  sample,  cystitis,  is  provided  in 
Appendix  C. 

4. 2. 3. 7 Program  Impact.  On  the  basis  of  the  strict  audit  criterion 
employed  herein,  the  present  findings  indicate  that  to  the  extent  that 
care  provided  in  direct,  total  compliance  with  an  algorithm  is  labelled 
as  "safe"  care,  very  few,  less  than  three  percent,  of  patients  treated 
solely  by  AMOSISTs  received  "safely"  delivered  care.  This  finding  is 
considered  even  more  profound  when  recognition  is  given  to  fact  that  this 
analysis  reflects  (a)  the  performance  of  AMICs  chosen  because  they  were 
judged  to  be  among  those  whose  operation  appeared  to  be  most  efficient 
and  most  in  accord  with  existing  program  guidelines,  and  (b)  the  per- 
formance of  AMOSISTs  dealing  with  a "simplest  case"  sample  comprised  of 
only  those  patients  for  which  a single  diagnosis  was  rendered. 

4.2.4  Field  Audit  Criteria 

4.2.4. 1 The  findings  which  have  been  cited  have  been  derived  from 
both  rather  strict  and  not-so-strict  criteria.  The  use  of  the  term 
"strict"  stems  from  the  fact  that  the  findings  resulted  from  the  rather 
rigid  application  of  an  overlay  transparency  to  each  DCS.  If  the  re- 
cordings evidenced  upon  the  DCS  were  not  in  strict  accord  with  the 
markings  appearing  in  the  overlay,  an  error  was  recorded.  On  the  other 


hand,  the  findings  are  considered  to  be  conservative  (derived  from  "not 
so  strict"  criteria)  because  no  effort  was  made  to  determine  if  the  en- 
tries pertaining  to  the  subjective  and  objective  findings  recorded  in 
the  upper  half  of  the  DCSs  were  compatible  with  some  diagnosis  other 
than  that  which  had  been  rendered.  However,  it  is  the  opinion  of  the 
writer  that  the  audit  process  employed  herein  was  much  more  rigorous 
than  that  encountered  during  Phase  I on-site  visits  to  any  of  the  AMICs 
(Schopper,  1978a).  In  contrast  to  the  audit  procedures  (or  lack  of 
same)  observed  during  these  visits,  those  presently  employed  might  be 
considered  unrealistic  in  their  demands  for  adherence  to  the  algorithm. 

Hence,  in  order  to  determine  if  this  was,  in  fact,  the  case,  a random 
sample  of  40  percent  of  the  DCSs  evaluated  were  provided  to  a former 
NCOIC  of  a large  AMIC  for  his  independent  evaluation. 

4. 2. 4. 2 When  the  DCSs  were  given  to  the  NCOIC,  he  was  informed  of 
the  concern  expressed  above,  and  he  was  asked  to  make  a gross  evaluation 
of  the  adequacy  of  each  of  the  DCSs  provided  to  him  according  to  the 
followir.g  three  categories:  (1)  Acceptable,  (2)  Marginally  Acceptable, 

(3)  Unacceptable.  He  was  not  given  criteria  with  which  to  define  each 
of  the  categories  cited  in  order  that  his  judgement  not  be  biased.  The 
results  of  his  independent  evaluation  of  the  117  DCSs  provided  to  him 
were  as  follows:  30.0  percent  were  judged  to  be  acceptable,  12.8  percent 
were  judged  to  be  marginally  acceptable,  and  a sizeable  57.2  percent 
were  judged  to  be  unacceptable.  Hence  it  appears  that  even  from  the 
prespective  of  a former  AMIC  NCOIC,  the  majority  of  the  DCSs  evaluated 
were  found  to  be  significantly  lacking  in  their  overall  adequacy. 

4.2.5  Additional  Findings:  DCS  Utilization. 

4.2.5. 1 Findings  which  relate  somewhat  less  obviously  to  the  issue 

of  the  safety  and  effectiveness  of  AMOSIST-delivered  care  are  those  which  pertain 
to  the  extent  to  which  the  AMOSISTs  employed  current  DCSs.  If  the  AMOSIST  does 
not  utilize  a DCS,  there  is  no  ready  manner  to  determine  whether  or  not 
he  performed  in  accordance  with  the  algorithm  that  he  was  supposed  to 
utilize.  Furthermore,  HSC  guidelines  (APC  Model  #13,  1976)  indicate  that 
the  AMOSIST  is  to  initiate  a DCS  for  every  patient  he  treats.  The  find- 
ings relevant  to  this  issue  based  upon  an  analysis  of  the  905  DCSs 
utilized  in  the  retrospective  patient  return  rate  evaluation  (paragraph 
4.3)  are  depicted  in  Table  6. 

4. 2. 5. 2 With  the  exception  of  treatment  for  injuries  addressed  by 
the  Musculoskeletal  (MS)  extremity  algorithm,  it  is  observed  that  DCSs 
were  not  initiated  for  between  12.8  percent  and  18.2  percent  of  the 
patients  treated  by  AMOSISTs.  (The  findings  for  the  excepted  MS, 

Extremity  algorithm  showed  much  higher  compliance,  i.e.,  for  only  4.8 
percent  of  the  patients  was  a DCS  not  initiated.)  Overall,  DCSs  were 
not  employed  for  13.2  percent  of  the  patients  treated  by  AMOSISTs. 

Because  all  patients  treated  by  AMOSISTs  have  been  included;  i.e.,  both 
those  for  which  the  AMOSIST  sought  physician  consultation  (which  are, 
therefore,  exempted  from  the  constraints  of  the  algorithm)  and  those  for 
which  the  AMOSIST  completed  treatment  without  consulting  a physician. 
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Table  6 


UTILIZATION  OF  CONTEMPORARY  ALGORITHM  DATA  COLLECTION  SHEETS  * 


TOTAL  NO. 

ALTERNATE 

FORM  UTILIZED 

ALGORITHM 

PATIENTS 

SF  600 

OLD  DCS 

TOTAL 

Eye  (1) 

21 

14.3% 

4.8% 

19.1% 

Respiratory  (2) 

450 

12.8% 

2.2% 

15.0% 

Gastrointestinal  (3) 

64 

15.6% 

7.8% 

23.3% 

Genitourinary  (4) 

104 

13.5% 

3.8% 

17.3% 

Musculo-Skeletal, 

Spine  (5) 

7 

14.3% 

0.0% 

14.3% 

Musculo-Skeletal , 
Extremity  (6) 

84 

4.8% 

9.5% 

14.3% 

Skin,  Regional  (7) 

88 

18.2% 

0.0% 

18.2% 

Skin,  General  (8) 

87 

16.1% 

0.0% 

16. 1% 

All  (1-8) 

905 

13.2% 

3.2% 

16.4% 

* A new  series  of  data  collection  sheets  had  been  authorized  for  use 
approximately  two  months  prior  to  the  beginning  of  the  study.  The 
sites  participating  in  the  study  were  informed  by  the  distributor 
of  these  new  versions  that  they  were  to  continue  to  use  the  existing 
forms  until  the  data  collection  for  the  study  had  been  completed. 
Hence,  the  term  "contemporary"  means  the  DCSs  that  were  depicted  in 
the  July  1975  version  of  the  AMOSIST  Manual. 
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one  might  be  tempted  to  argue  that  these  findings  are  not  appropriate 
ones  with  which  to  address  these  issues  of  safety  and  effectiveness  of 

care.  However,  this  argument  is  fallacious,  for  it  implies  that  at  the  1 

outset  of  treatment  (the  time  at  which  an  AMOSIST  must  make  a choice 

regarding  the  form  upon  which  he  is  to  record  his  findings)  the  AMOSIST 

can  differentially  detect  apriorl  which  patient's  complaints  are  going 

to  require  that  he  consult  with  a physician  before  he  has  completed  his 

treatment. 

4. 2. 5. 3  Table  6 also  portrays  additional  related  data;  i.e.,  that 
of  the  extent  to  which  AMOSISTs  employed  outdated  DCSs.  Relative  to  the 
frequency  with  which  SF  600s  are  employed  in  lieu  of  DCSs,  the  use  of 
) outdated  DCSs  (3.2  percent  overall)  is  observed  to  be  much  less  of  a 

problem.  Nonetheless,  the  continuing  use  of  old  DCSs  does  pose  a po- 
tential care-related  problem  if  the  contemporary  DCSs  address  data  which 
are  not  cited  within  previous  editions.  Too,  even  if  no  additional  con- 
tent is  addressed,  the  change  in  format  which  occurs  requires  that  the 
individual (s)  performing  the  audits  of  the  DCSs  be  familiar  with  more 
than  one  format  if  that  function  is  to  be  performed  efficiently. 

4.2.6  Additional  Findings:  Frequency  of  Occurrence  of  Categories 
of  Illness  and  of  Diagnoses . 

4.2.6. 1 General.  Information  which  is  seemingly  extraneous  to  the 
general  purpose  of  this  report  is  that  which  pertains  to  (a)  the  fre- 
quency with  which  the  eight  general  algorithms  are  utilized  and  (b)  the 
frequency  with  which  the  diagnoses  contained  in  the  AMOSIST  Manual  are 
rendered.  The  905  DCSs  used  in  the  evaluation  of  patient  return  rates 
(paragraph  4.3)  was  utilized  as  the  sample  for  the  present  evaluation. 

The  relevance  of  the  present  findings  of  the  issue  of  "safety-of-care" 
is  discussed  »fter  the  findings  have  been  described. 

4. 2. 6. 2 Algorithms.  The  findings  depicted  in  Table  6 clearly  show 
the  Musculo-Skeletal,  Spine  algorithm  to  be  that  which  was  least  utilized. 

Less  than  1 percent  (0.78  percent)  of  all  patients  treated  by  a single 
algorithm  employed  this  algorithm.  Relative  to  most  remaining  algorithms, 

(which  ranged  in  utilization  from  6.7  percent  to  47  percent),  the  Eye 
algorithm  was  also  infrequently  utilized.  Only  2.2  percent  of  the  905 
patients  in  the  sample  being  discussed  were  seen  for  eye  problems. 

4. 2. 6. 3 Diagnoses.  Within  the  various  algorithms  there  were  also 
specific  diagnoses  which  were  very  infrequently  encountered.  All  diag- 
noses encountered  two  or  less  times  are  cited  in  Table  7.  It  is  apparent 
that  a substantial  portion  of  the  diagnoses  cited  in  some  of  the  algo- 
rithms— particularly  the  Respiratory,  Genitourinary,  MS  Extremity,  and 
Skin-Regional  algorithms — are  little  used.  Overall,  it  is  noted  that  20 
(38.5%)  of  the  52  possible  specific  diagnosis  contained  in  the  existing 
AMOSIST  Manual  have  occurred  two  or  less  times  in  the  present  sample  of 
905.  Nine  (17.3%)  never  occurred. 

4. 2. 6. 4 The  findings  depicted  in  Table  7 indicate  that  only  13 
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Table  7 


DIAGNOSES  INFREQUENTLY  ENCOUNTERED  IN  THE  PRESENT  RETROSPECTIVE 
AUDIT  OF  905  AMOS I ST-TREATED  PATIENTS 


ALGORITHM  DIAGNOSES  (NUMBER  OF  TIMES  ENCOUNTERED  *) 


Eye  (1) 

Blepharitis  (0),  Chalazion  (0),  Subjunctival 
Hemorrhage  (0) 

Respiratory  (2) 

Toxic  Labyrinthitis  (0),  Furuncle  (0),  Chronic 

Otitis  Externa  (0),  Streptococcal  Sore  Throat  (1), 
Acute  Laryngitis,  Non-specific  (lm) 

Gastrointestinal  (3) 

None  (Only  two  specific  diagnoses  are  renderable; 
both  used) 

Genitourinary  (4) 

Genitourinary  Gonorrhea  (0),  Urethritis,  Non- 
specific (1),  Vaginitis,  Trichomones  (lm), 

Vaginitis  Candida  (1,1m), 

Musculo-Skeletal , 

Spine  (5) 

Other  (0) 

Musculo-Skeletal, 
Extremity  (6) 

Bicipital  Tenosynovitis  (0),  Supraspinatus 

Tendinitis  (2),  Ankle  Sprain  (2) 

Skin, Regional  (7) 

Tinea  Cruris  (0),  Pediculosis  Pubis  (1),  Tinea 

Pedis  (1,1m),  Pseudofolleculitis  (2) 

Skin, General  (8) 

Tinea  Corporis  (0) 

* The  letter  "m"  appearing  as  a subscript  indicates  that  the  diagnosis 
appeared  in  conjunction  with  one  or  more  other  diagnoses  addressed 
within  the  algorithm. 
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of  905  patients  were  treated  for  illnesses  included  among  the  20  diagno- 
ses cited.  This  suggests  that  a substantial  portion  of  the  AMOSIST  Man- 
ual, nearly  AO  percent  of  the  52  specific  diagnoses  contained  within  it, 
addresses  diagnoses  which  are  applicable  to  but  a small  portion  of  the 
patients  seeking  treatment.  One  implication  of  these  findings  is  that 
the  AMOSIST  Manual  is  inefficient  to  utilize.  As  a consequence,  it  is 
likely  that  such  inefficiency  has  contributed  substantially  to  the 
AMOSIST' s defacto  decision  not  to  utilize  the  AMOSIST  Manual,  a decision 
which,  in  turn,  has  adversely  affected  the  "safety-of-care"  aspect  of  the 
program.  The  rationale  is  similar  to  that  previously  employed  in  dis- 
cussing the  apparent  unwillingness  of  AMOSISTs  to  record  vital  signs; 
i.e.,  a tendency  (not  confined  to  AMOSISTs)  to  disengage  from  tasks 
which  have  little  impact  upon  their  work  and  require  increased  amounts  of  time 
and  effort  for  their  accomplishment.  The  implicit,  unverbalized  question  which 
is  believed  to  have  been  posed  and  answered  by  the  AMOSIST  is  "Why  should  I 
continue  to  regularly  employ  a series  of  instructions  when  nearly  40 
percent  of  those  instructions  have  no  impact  upon  my  ultimate  decision 
and  course  of  action?  To  do  so  is  not  only  boring  and  frustrating,  but 
it  is  generally  nonproductive."  The  answer  which  is  behaviorally  ex- 
pressed is  clear:  As  indicated  in  the  initial  report  (Schopper,  1978a), 
they  do  not  use  it.* 

4. 2. 6. 5 The  decision  not  to  use  the  AMOSIST  Manual  leaves  the 
AMOSIST  with  two  alternatives,  neither  of  which  is  desirable:  either 
rely  on  his  own  memory  (a  tactic  which  the  reults  of  the  present  evalu- 
ation has  shown  to  be  ill  advised)  or  rely  on  the  physician  by  increasing 
the  frequency  of  requested  consultations  (a  tactic  which  is  not  cost 
effective).  This  circumstance,  the  existence  of  a substantial  number  of 
rarely  used  instructions,  is  also  one  that  is  likely  to  have  had  a 
similar  negative  impact  on  any  effort  to  invoke  regularly  employed,  for- 
mally documented  audit  procedures  since  the  auditor  would  be  similarly 
burdened.  This  absence  of  use  of  the  AMOSIST  Manual,  the  AMOSISTs'  pri- 
mary medical  reference,  by  both  the  AMOSISTs  and  those  who  perform  the 
audits,  is  believed  to  be  a principal  contributor  to  their  inability  to  de- 
liver error  free  (as  defined  herein  "safe")  medical  care.  Since  the  sole 
impact  of  eliminating  entirely  these  20  rarely  used  diagnoses  would  be  that 
of  a very  small  increase  (.56%)  in  the  proportion  of  AMOSIST-treated  patients 


* As  cited  in  the  initial  report,  another  substantial  factor  contributing 
to  the  lack  of  utilization  of  the  AMOSIST  Manual  is  the  AMOSIST^  aversion 
to  making  overt  reference  to  it  while  treating  the  patient — an  action 
which  he  feels  conveys  a sense  of  ineptitude  to  the  patient. 
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requiring  a physician's  consultation  in  order  to  complete  their  treatment,* 
a need  to  reexamine  the  range  and  quantity  of  illness  specifically  ad- 
dressed in  the  AMOSIST  Manual  is  clearly  indicated.** 

4.2.7  DCS  Error-Rate:  Comparison  with  BAMC /University  of  Washington 
Project . 


4.2. 7.1  General.  The  results  of  the  DCS  audits  presented  herein 
are  not  particularly  laudable.  Of  the  entire  sample  of  282  records 
audited,  only  8 (2.8%)  were  entirely  devoid  of  error.  This  is  in  sharp 
contrast  to  the  results  which  are  reported  to  have  been  obtained  from 
the  AM1C  at  BAMC.***  Therein,  between  75  and  85  percent  of  the  DCSs  for 
each  of  the  algorithms  which  have  been  investigated  there  (URI,  headache, 
back  pain,  and  pediatric  URI)  have  been  error-free.  While  the  actual 
algorithms  employed  at  the  BAMC  do  differ  from  those  employed  at  AMICs 
in  other  Army  hospitals,  it  is  unlikely  that  these  differences  account 
for  the  entire  difference  observed  in  the  error  rates  between  these  two 
systems.  The  discussion  which  follows  summarizes  several  of  the  factors 
which  are  likely  to  contribute  to  the  existing  disparity. 


* The  0.56  percent  increase  cited  assumes  that  the  base  rate  of  requests 
for  physician  consultations  (62  percent  in  the  overall  sample  of  905 
being  considered)  would  be  applicable  to  the  portion  of  '■he  sample  (the 
13  of  the  905)  being  discussed.  The  actual  increase  in  the  proportion 
of  patients  requiring  a physician  consult  as  a result  of  the  deletion 
of  these  20  diagnosis  would  be  only  that  proportion  of  the  entire  de- 
leted percentage  (13  of  905,  1.44%)  in  excess  of  62  percent,  i.e., 

38%  of  1.44%  or  0.56%. 

**  While  a need  to  reconsider  the  scope  of  the  AMOSIST  Manual  from  the 
viewpoint  of  the  relative  utility  of  the  diagnoses  included  is  clear, 
the  existence  of  a possible  seasonal  bias  of  unknown  nagnitude  in  the 
present  findings,  it  would  be  appropriate  to  examine  the  issue  more 
fully  prior  to  undertaking  a revision.  While  it  is  the  case  that  the 
present  sample  included  not  only  the  single  diagnosis  DCSs,  but  also 
the  multiple  diagnosis  DCSs  (inclusive  of  those  for  which  as  many  as 
three,  four,  and  occasionally  five  diagnosis  were  indicated),  it  is  none- 
theless, noted  that  the  data  from  which  these  findings  were  derived  were 
obtained  from  only  two  sites  and  were  collected  during  a three  and  one- 
half  month  period  from  mid-February  through  May.  Nevertheless,  the  pre- 
sent findings  rather  strongly  suggest  that  the  scope  of  the  illnesses 
addressed  is  too  large. 

***  Too,  civilian  care  providers  (MEDEXs)  trained  at  a level  equivalent  to 
the  US  Army's  Physician  Assistant  (PA)  have  utilized  similar  algorithms 
and  have  been  rjported  to  have  performed  at  a 47  percent  error-free  level 
of  proficiency  in  a study  of  3024  patients  involving  algorithms  for  11 
type  of  illnesses  (Sox  et  al,  1973). 


4. 2. 7.2  DCS  Format.  The  URI  DCS  utilized  at  BAMC  is  depicted  in 
Figures  2 and  3.  This  is  contrasted  with  the  URI  DCS  utilized  by  the 
sites  studied  at  the  time  of  the  data  collection  for  the  present  study 
(Figure  1)  and  the  URI  DCS  which  is  presently  being  utilized  at  all  Army 
AMICs  other  than  the  one  at  BAMC  (Figure  4).  Albeit  the  more  recent 
version  of  the  non-BAMC  DCS  appears  better  organized  and  more  aestheti- 
cally appealing  than  its  predecessor,  the  content  is  not  changed.  Its 
sole  purpose  remains  as  that  of  a checklist,  a form  which  serves  merely 
as  an  organized  vehicle  upon  which  to  collect  the  clinical  data.  This 
is  in  considerable  contrast  to  the  BAMC  URI  DCS  which  serves  multiple 
purposes.  On  this  DCS,  not  only  are  places  provided  to  record  the 
findings,  but  it  also  concisely  depicts  the  algorithm  to  be  followed 

on  the  reverse  side  (Figure  3) . One  principal  advantage  which  derives 
from  this  format  is  the  elimination  of  the  need  to  utilize  a separate 
reference  manual  during  the  actual  course  of  treatment.  As  indicated  in 
the  initial  report  of  this  series  (Schopper,  1978a),  AMOSISTs  working  at 
the  sites  evaluated  in  the  present  study  were  found  to  have  avoided  the 
overt  utilization  of  their  AMOSIST  Manuals.  At  no  time  was  an  AMOSIST 
Manual  observed  on  the  top  of  an  AMOSIST' s desk.  Further,  it  was  found 
that  AMOSISTs  occasionally  were  unable  to  locate  their  manual  at  all  on 
the  hospital  premises.  The  impression  obtained  from  speaking  with 
AMOSISTs  about  the  matter  was  that  their  avoidance  of  the  AMOSIST  Manual 
was  due  more  to  their  desire  not  to  appear  dependent  upon  a manual  in 
the  eyes  of  their  patients  than  from  their  perceptions  that  their  know- 
ledge was  such  that  they  no  longer  needed  to  consult  it.  Clearly,  by 
printing  the  algorithm  directly  upon  the  same  form  utilized  to  record 
the  data,  the  apparent  dependence  of  the  AMOSIST  upon  the  guidance  pro- 
vided by  an  algorithm  is  much  less  obvious  to  a patient,  and  the  aversion 
to  employing  same  is  likely  to  be  markedly  decreased. 

4. 2. 7. 3 Algorithm  Simplicity. 

4. 2. 7. 3.1  Aside  from  the  differences  in  formats  associated  with  the 
DCSs  themselves,  another  factor  which  may  have  contributed  to  the  gross 
disparity  in  error  rates  is  that  the  BAMC  algorithm  is  easier  to  under- 
stand and  utilize.  From  a global  perspective,  it  is  the  writer's  opinion 
that  this  is  true,  at  least  as  regards  the  algorithm  that  is  most  fre- 
quently employed,  the  URI  algorithm.  While  no  independent  data  have  been 
gathered  to  formally  evaluate  this  possibility  (e.g.,  as  by  having  a 
group  of  physicians  or  AMOSISTs  independently  rate  the  two  algorithms  on 
the  dimension  of  "ease-of-utilization") , some  readily  documentable  obser- 
vations appear  to  offer  some  support  for  the  writer's  contention.  The 
first  is  that  the  number  of  possible  diagnoses  which  may  be  rendered  is 
markedly  smaller  for  the  BAMC  algorithm  than  for  its  counterpart  (See 
Figures  2 and  4).  Only  seven  specific  conditions,  plus  the  alternative 
"other,"  are  indicated  on  the  BAMC  DCS  (see  lower  right  hand  corner  of 
Figure  2 under  the  subheading  "TREATMENT  PROTOCOLS").  In  comparison, 
the  non-BAMC  DCS  cites  16  specific  diagnoses  and  "Other"  (see  middle  of 
Figure  4 under  the  heading  "ASSESSMENT") . While  the  list  of  illnesses 
addressed  by  the  AMOSIST  Manual  is  clearly  longer,  longer  does  not 
necessarily  mean  better.  The  findings  of  the  present  study  (discussed 
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Figure  2 


CHECKLIST  SIDE  OF  DATA  COLLECTION  SHEET  USED  AT  BROOKE  ARMY 
MEDICAL  CENTER 


New  Visit 

URr  CHECKLIST  (#13)  Return  Visit  (within  2 weeks) 


AMQSISTS:  Mark  ALL  Findings  /^Present,  O-Absent  (absence  of  mark  means  item  wasn't  examined) . 
Physician  Disagrees  with  Presence  or  Absence  of  a Finding  or  Plan. 

SUBJECTIVE  DATA: 

Chief  Complaint: Medication(s): 


Duration 


Jdays): 


__1  1-7  days 
2 8-14  days 


_3  15-21  days 
4 >21  days 


Allergy(s) 


7 Sore  throat/swallowing  pain 

_8  Hx  strep  exposure 

9 S/P  tonsil  lectorgy 

10  Facial/dental  pain 

11  Runny/stuffy  nose 

12  Hx  shaking  chills 

13  Cough  presen t- 

14  Productive 

15  Chronic  (6  mo/2  yr) 

OTHER  HISTORY: 


Age: 


_5  Age  12-35  years  6 Age  > 55  years 


_17  Pneumonia  in  past  5 yr 
~18  Chest  pain- 

19  Pleuritic 

20  Precordial/Substernal/ 

Exertional 

_21  Smokes  cigarettes- 

22  <20  pk-yrs 

23  20-40  pk-yrs 

24  >40  pk-yrs 


Right/Left  Ear  Symptoms- 
_/_ 25  Pain 
/ 26  Tinnitus 
/ 27  Loss  of  hearing 
__?8  CHronlc  Diseases- 

90  Heart  Disease 

91  Kidney  Disease 

92  Diabetes  (on  medication) 

93  Asthma/Lung  Disease 

95  Blood  Disease 

99  Other: 


OBJECTIVE  0ATA: 


Temp:  _29  > 101UF|  jResp: 


30  > 30/mi  n 


31  Tonsils  present 
'32  Abnormal  throat  exam- 
33  Tonsils  swollen 

34  Red  throat 

35  Exudate  present 

36  Lymphoid  hyperplasia 

37  Other: 

J ST  Abnormal  nasal  exam- 

" 39  Purulent  discharge 

40  Non-pur.  discharge 

41  Other: 

Tl  Sinus  tenderness : 

Rlght/Left 

/ frontal 


_43  Ant.  cervical  nodes  enlarged 
_44  Post,  cervical  nodes  enlarged 

as  Spleen  palpable  (HD  data) 

_46  Abnormal  chest  exam- 

47  Localized  chest  findings 

48  Rales: 

49  Accessory  muscles  used 

50  Poor  expansion,  unilateral 

51  Dullness: 

52  Chest  wall  tender: 

53  Hyperresonance 

54  ♦ Breath  sds: 

55  Wheezes: 


Pulse: 


BP(optional) / 

56  Abnormal  ear  exam- 
’ Right/Left  Sided 

57  58  Foreign  body/wax  present 

59  60  Obstruction  after  attempted 

removal 

61  62  Ext.  canal  inflanrred/pus 

63  64  Tympanic  membrane  obscured 

65  66  TM  abnormal 

67  68  TM  red/bulging 

69  70  TM  loss  of  light  reflex 

71  72  TM  loss  of  landmarks 

“ >3  “24  TM  air-fluid  level 
75  76  Scarred  TM 

77  78  Other  ear  abnormality 


/ M axillary 

OTHER  PHYSICAL: 


PLANS: 

103  Throat  culture 
“109  WBC  with  differential 

110  Mono  spot 

111  Chest  xray 

149  Other  fper  MD  - V) 

RESULTS: 

79  Pos.  for  B-strep 

80  Atypical  lymphs  present 

81  Mono  spot  positive 

82  Infiltrate  present 

TREATMENT  PROTOCOLS: 

150  A 205  B 206  Strep  throat  antibiotic 

151  Sympt.  sore  throat 

152  A 207  B 203  Runny  nose 

153  A 209  B 210  C 211  Acute  sinu-itis  antibotic 

154  Otitis  externa 

155  A 212  8 213  C 214  Otitis  Media 

167  A 215  B 216  Cough 

217  A 218  B 219  C 220  Pneumonia 

175  Other  (per  HD  - : 

INITIAL  ASSESSMENT: 

PHYSICIAN  INVOLVEMENT: 

102  MO  examined  patient 

221  MD  verbally  consulted 

222  MD  not  involved 

PATIENT  IDENTIFICATION: 

DATE 

SIGNATURES: 

AMIC  t 

AMOSIST 

AMOS  1ST  it 

M.D. 

(reduced  size) 

TIME  IN 

TIKE  OUT 
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ALGORITHM  SIDE  OF  URI  DATA  COLLECTION  SHEET  USED  AT  BROOKE  ARMY  MEDICAL  CENTER  (BAMC) 


Cough  protocol  MO  verbol  consult  ~1  y** J Consult  MD 

recommended 


Figure  4 


LATEST  VERSION  OF  RESPIRATORY  DATA  COLLECTION  SHEETS  USED  AT 
US  ARMY  MEDDACS 


ACADEMY  OF  HEALTH  SCIENCES 
MED  & SURG  DIV  AMOSIST  SEC 


M 28-300-015-1 

048 


RESPIRATORY  - chief  comflaintisi  a duration 

TIME  SEEN 

DATE: 

TIME  OUT 

SUBJECTIVE  DATA 

OBJECTIVE  DATA 

Chtaf  complaint  corract 

# Return 

risit  for  unchangad 

TEMP  PULSE  REST 

BP 

PomJbty  prt*n»nt 

• 9wkin«  chills 

* - . Abnormal  vital  signs 

• 

Appears  severely  III 

* F*v«r  for  mon  than  6 d 

* Rah  lor  Isa  that  6d 

THROAT  & NECK 

EAR 

* Taking  MD  ordered  mod: 

# Contraindkotad  mad- 

Rad  pharynx 

- 

^•poid  tandamass 

Tons* Ur  exudate 

a 

_ aavara 

THROAT  A NECK 

EAR 

* ParrtowaiSar  mailing 

- 

Ear  drainoga 

Poo  TC  In  la*  lOd 

* aavara 

Lymph  nodaa 

bloody 

Vartioo 

* aavara 

* ptxuHor 

white  • y allow 

_ hd  hx  (twum  favor 

Earacha 

for  days 

aandar  interior 

- 

TM  nm  .sibi. 

canal  fill  with  wax 

Ho>mrw  for: 

Ear  hchmg 

HOSE  5 SINUS 

— 

TM  toss  of  landmarks 

NOSE  B>  SINUS 

CHEST 

watary 

TM  air  fluid  treat 

Nml  confMtlon 

* Short  naas  of  braath 

purulent 

TM  bubbUs 

final  drainaga 

Cough  for  days 

a 

_ purulont 

__  not  Improving 

* 

TM  scar 

racurrom/n— oml 

♦ productwa/poculant 

* Fac id  sevail  mg 

a 

TM  parforatad 

watery 

* bloody  aputum 

Sinua  tandarnasa 

Canal  tandamaas 

Itching  «yn/non 

Chest  pain  mild 

Taaring 

CHEST 

local 

# S3  no*  x-ray  In  last  3 moa 

OTHER 

# * Unequal  braath  aounds 

• 

pointing 

....  Runny  noaa  for: 

Haodacha  for: 

** Rale*.  rhon  or  whaeamg 

outar  half 

OTHER: 

*r  Chaat  point  tandomaas 

— 

Ear  fhauras/sralss 

OTHER 

#*  Dacraaasd  naefc  r tax  ion 

OTHER: 

ASSESSMENT 

Labyrinth  It  la 

Acuta  sinusitis 

Laryngrtk  a 

OTHER 

than  AMOSIST  Manual: 

Acuta  otitis  madia 

AJIargic 

rhinitis 

Costochondritis 

Acuta  ssroua  otitis  madia 

Rhinitis. 

NOS 

Bronchitis 

Furunda 

Strap  sora  throat 

Baadachs 

Acuta  otitis  sxtama 

Anita  pharyn,  poss  strap 

Chronic  otitis  sxtama 

Acuta  pharyngitis.  NOS 

TREATMENT 

PLAN 

* Tamporery  profHa  for: 

DRAM  AMINE  (Dimanhydrmatal  50  mg  # 12.  1 cap  06  H 

*-  — Quartan  for: 

CHLORTRIMETON  4 mg 

# 30, 

1 tab  Q6M  prn 

Ratum  to  dinie  on: 

ACTIFED  #30.  1 tab 

TID  prn 

PEN  VK  250  mg  #40. 

1 tai'  QID 

xIOd 

DIMETAPP  Extan  tabs  #20.  1 

tab  BID 

ERYTHROMYCIN  250  mg 

#40.  t tab  QID  xIOd 

NEOSYNEPHRINE  Sol  a 25%  15cc.  2 ggt  QID  prn 

CORTISPORIN  Otic  Solution  10cc.  4 ggt  OID  xIOd 

* other  th.n  AMOSIST  MUmwl: 

HYDROCORTISONE  Craam 

1%  30  Gm 

apply  BID 

ASPIRIN  325  mg  # 40. 

2 tabs  Q6H 

pm 

TYLENOL  325  mg  # 40. 

2 tabs  06H 

pm 

CEPACOL  Loianga  # 24. 

1 lot  Q4H 

pm 

ROBITUSSIN-DM  120ec. 

1 tap  Q4H  pm 

AODRESSOGRAPH  or  Nama.  FPN 

SSN.  DOB. 

consultations 

8.  REFERRALS 

DISPOSITIONS 

Sax,  Phona  Numbar.  MTF,  Status 

Throat  culture  Ear  culture 

Duty  or  Horn* 

Ear  canal  irrigation  Monospot 

* 

Hospital  ward: 

Sinus  x-rays 

Nos*  culture 

• 

Chart  x-ray 

AMOSIST 

MEDICAL  DOCTOR 

♦ - POS  / R • 

♦ rignt  si  da  / Rv.  • ♦ bott>  n<3*s 

/ a - 

MD  trana'ar  If  ♦ (POS)  [ 

O - NEC  / L 

■ ♦ left  tida  / # • do  onty  If  r 

sQuastad 
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earlier,  paragraph  4.2.6. 3,  Table  7)  suggest  that  the  present  URI 
algorithm  contains  "excess  baggage"  in  that  six  of  the  16  specific  diag- 
noses were  observed  to  have  occurred  either  not  at  all  or  only  once 
among  450  URI  cases  evaluated.  (These  data  included  both  Sw.:.gle  and 
multiple  diagnoses  occurring  among  the  DCSs  evaluated  in  the  retro- 
spective DCS  sample.)  In  contrast  to  this,  seven  of  the  eight  conditions 
addressed  by  the  BAMC  algorithm  evidence  an  incidence  ranging  from  6-29 
percent  among  the  URI  patients  treated.  Pneumonia  was  the  sole  BAMC  ex- 
ception, appearing  among  only  one  percent  of  the  patients;  nevertheless, 
this  rate  is  approximately  four  and  one-half  times  that  evidenced  among 
the  most  frequently  occurring  of  the  six  "little-utilized"  diagnoses  of 
the  non-BAMC  algorithms.  Stated  somewhat  differently,  the  findings  of 
the  present  study  suggest  that  the  number  of  specific  diagnosis  addressed 
in  the  URI  algorithm  of  the  AMOSIST  Manual  could  be  reduced  nearly  40 
percent  (from  16  to  10)  while  affecting  the  treatment  of  less  than  one- 
half  of  one  percent  (.44%)  of  the  patients  examined.  (The  term  "affecting" 
is  used  to  denote  simply  that  the  treatment  for  these  patients,  as  a re- 
sult of  their  not  having  received  a diagnosis  that  was  algorithm-directed, 
would  have  to  i /olve  a physician  consultation.) 

4. 2. 7. 3. 2 A finding  related  to  those  cited  above  concerns  the 
number  of  logic  blocks  addressed  in  the  algorithm.  The  complete  URI 
algorithm  appearing  in  the  AMOSIST  Manual  is  comprised  of  142  logic/ 
decision  blocks  (including  nearly  55  which  address  the  issue  of  medica- 
tions and  contraindications  for  same — nearly  one-half  of  which  are  con- 
cerned with  making  a choice  between  aspirin  and  acetaminophen) . The  BAMC 
algorithm  (Figure  2)  has  only  46  logic/decision  blocks. 

4. 2. 7. 3. 3 These  observations,  which  pertain  solely  to  the  URI 
algorithm,  are  clearly  consistent  with  those  made  earlier  regarding  the 
AMOSIST  Manual  as  a whole  and  serve  to  reinforce  the  previously  made 
assertion  that  there  exists  a need  to  revise  and  delimit  the  range  of  ill- 
nesses addressed  in  the  AMOSIST  Manual  and  attendant  DCSs  in  order  to  in- 
crease the  efficiency  of  the  program  and  decrease  the  likelihood  that  the 
algorithms  will  be  rejected  by  the  intended  user  as  the  result  of  boredom 
and  frustration  at  having  to  utilize  a high  effort,  low  yield  reference. 

4. 2. 7. 4 Personnel  Selection  and  Work  Environment. 

4. 2. 7.4.1  This  issue  has  been  more  thoroughly  discussed  elsewhere 
(Schopper,  1978a;  HSC  Memorandum,  1976).  As  it  pertains  herein,  it  is 
likely  that  some  portion  of  the  difference  observed  in  the  error  rates 
encountered  between  BAMC  AMOSISTs  and  those  encountered  in  the  present 
study  are  due  to  (a)  differences  in  the  personnel  selection  processes 
respective  to  each  of  the  systems  and  (b)  the  working  environment  which 
exists  at  each.  In  brief,  BAMC  is  one  of  the  Army's  major  medical  train- 
ing facilities  and  is,  therefore,  a facility  which  is  reasonably  well 
staffed  with  highly  qualified  personnel.  Furthermore,  the  AMIC  which 
exists  there  is  the  focus  of  a large  scale,  well  funded  clinical  research 
program  intended  to  develop  clinically  validated  algorithms  appropriate 
for  use  by  AMOSISTs.  The  AMOSISTs  and  the  AMIC  are,  therefore,  the  object 
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of  considerably  more  interest  and  attention  than  typically  exists  at  most 
other  sites.  The  AMOSISTs  who  work  there  go  through  a thorough  screening 
process  to  select  the  more  capable  individuals.  Additionally,  if  attri- 
tion rate  can  be  taken  as  a valid  indication,  the  training  demands  which 
exist  and  the  criteria  employed  for  success  in  the  AMOSISTs'  "OJT"  train- 
ing phases  are  much  more  rigorous  at  BAMC  than  at  any  other  facility. 

The  attrition  rate  (42%  during  the  past  fiscal  year)  among  BAMC  AMOSIST 
candidates  during  their  OJT  phases  is  markedly  higher  than  that  evidenced 
at  any  other  facility.  BAMC  AMOSISTs  are  also  subject  to  closer  daily 
monitorship  than  is  present  at  any  other  AMIC.  They  receive  regular, 
daily  computerized  feedback  and  monthly  individual  counseling  sessions 
regarding  their  performance.  Furthermore,  their  continuation  in  the  pro- 
gram is  contingent  upon  their  performance  and  the  mutual  consent  of  the 
AMOSIST  and  the  supervising  physician.  There  exists,  therefore,  a 
definite  challenge  in  their  work.  Additionally,  any  tendency  toward 
boredom  is  counteracted  to  some  degree  by  the  modification  of  DCSs  which 
occasionally  takes  place  as  the  algorithms  are  examined  and  evaluated. 

4. 2. 7. 4. 2 The  circumstances  cited  above  are  in  contrast  to  the 
situation  which  exists  elsewhere.  At  many  other  facilities  the  AMOSIST 
concept  is  less  highly  regarded.  Auditing  procedures  are  often  elemen- 
tary, performance  related  feedback  is  irregular  and  AMOSISTs  make  little 
direct  use  of  the  AMOSIST  Manual  (Schopper,  1978a).  It  is  not  infre- 
quently the  case  that  potential  candidates  for  AMOSIST  positions  suffer 
a somewhat  negative  bias  because  of  the  unwillingness  of  existing  services 
to  release  good-pef orming , highly  motivated  personnel  so  that  they  may 
be  considered  for  selction  to  be  trained  as  an  AMOSIST.  Additionally, 
due  perhaps  to  the  apparent  shortage  of  corpsmen  in  general  (and  to  the 
investment  in  time  that  has  been  made  to  obtain  resident  AHS  instruction) , 
most  sites  seem  to  be  reluctant  to  impose  high  standards  and  submit 
negative  Phase  II  or  Phase  III  OJT  evaluations  on  their  AMOSIST  trainees. 

4.2. 7.5  Relative  Importance  of  Factors:  Some  Soft  Data.  Several 
factors  have  been  cited  as  being  possible  contributors  to  the  disparity 
between  BAMC  error  rates  and  those  evidenced  in  the  present  study.  Others, 
too,  may  be  contributors  e.g.,  BAMC's  results  for  adult  patients  are  con- 
fined to  only  three  algorithms  whereas  the  present  data  have  addressed 
all  eight  algorithms  employed  by  US  Army  MEDDACs.  While  there  exists  no 
formal  data  with  which  to  assess  the  relative  magnitudes  of  the  contri- 
butions ascribable  to  each  of  these  components  in  attempting  to  account 
for  the  observed  difference,  the  implication  of  some  "soft"  data  mav  be 
highly  significant.  When  speaking  to  some  of  the  BAMC  AMOSISTs  about 
their  work,  it  was  clear  that  the  daily  prospect  of  facing  a computerized 
printout  which  critiqued  their  work  of  the  previous  day,  and  knowing  that 
they  were  surrounded  by  physicians  and  researchers  who  were  privy  to  and 
equally  as  interested  as  they,  themselves,  in  those  findings,  combined 
to  act  as  a profound  motivating  influence — one  that  made  them  adhere  to 
their  algorithms  very  closely.  Although  the  computerzied  output  does, 
no  doubt,  lend  a particularly  impressive  aspect  to  this  procedure,  it 
is  the  writer's  viewpoint  that  similar  results  could  be  achieved  by  the 
regular  and  frequent  presentation  of  the  results  of  any  meaningful  audit 
in  an  informative  manner  by  a significant  supervisor  (e.g.,  one's  rater 
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or  indorser)  who  shows  real  interest  in  the  data  and  like  concern  for  the 
maintenance  of  high  standards.  Though  the  contribution  of  "meaningful 
data"  to  the  success  of  such  a process  is  unquestioned,  it  is  emphasized 
that  the  frequency  of  feedback  and  the  role  of  the  supervisor  are  also 
factors  of  considerable  importance.  That  this  is  so  is  suggested  by 
observations  from  studies  being  conducted  by  BAMC  and  the  Univer- 
sity of  Washington.  An  example  of  the  importance  of  the  supervisor's 
role  is  the  occurrence  of  some  deterioration  in  the  performance  of  BAMC 
AMOSISTs  as  the  result  of  a brief  period  during  which  the  BAMC  AMIC  came 
under  the  direction  of  a less-than-enthusiastic  physician  (personal 
communication,  Wolcott,  1978).  The  importance  of  having  frequent  feed- 
back is  supported  by  data  from  the  recent  algorithm  research  addressing 
troop  medical  clinic  sick  call  procedures  at  Fort  Hood,  Texas  (Active 
Duty  Troop  Medical  Clinic  (ADTMC)  Project,  Wolcott,  1977)  and  at  the 
Academy  of  Health  Sciences,  Fort  Sam  Houston,  Texas.  Unpublished  data 
from  this  study  show  that  irregular  and,  at  times,  infrequent  feedback 
regarding  the  performance  of  corpsmen  using  a relatively  simple  triaging 
algorithm  had  little  impact  and  yielded  error  rates  averaging  32  percent 
over  a three  month  trial  period.  However,  the  institution  of  a system  of 
daily,  immediate  case-by-case  feedback  lowered  the  error  rate  to  2.5  per- 
cent within  nine  weeks. 

4. 3 Patient  Return  Rates. 

4.3.1  General. 

4.3. 1.1  The  data  discussed  in  the  previous  sections  addressed  a 
function,  adherence  to  algorithms,  which  was  unique  to  the  AMIC  and 
focussed  upon  as  the  measure  of  "safety-of-care"  employed  in  the  study. 

As  a result  of  the  uniqueness  of  the  function  it  was  not  possible  to  make 
any  comparisons  with  GOC  performance.  The  measure  which  addressed  medical 
"ef fectiveness-of-care, " that  of  the  non-directed  patient  return  rates, 
does  permit  a be tween-clinic  comparison,  however. 

4.3. 1.2  As  employed  in  the  present  study  the  term  "patient  return 
rate"  is  operationally  defined  as  any  non-directed  return  to  a medical 
facility  to  obtain  additional  treatment  for  an  illness  or  medical  com- 
plaint for  which  he  had  previously  received  care  in  an  AMIC  or  GOC.  For 
the  purpose  of  the  present  study,  the  return  visit  had  to  occur  within 
two  weeks  of  the  date  of  the  initial  treatment  in  the  AMIC  or  GOC.  The 
global  instruction  "p.r.n."  (pro  re  rate:  as  needed,  as  desired)  was  not 
considered  as  an  instruction  to  return  for  the  present  study;  hence,  any 
patient  returns  encountered  wherein  the  initial  visit  contained  these 
instructions  were  counted  as  non-directed  returns.  The  comparison  ad- 
dressed only  those  patients  treated  for  illnesses  which  were  classifiable 
among  eight  categories  of  illness  addressed  by  the  AMOSISTs  algorithms. 

4.3.2  GOC  versus  AMIC  Comparison.  The  return  rates  evidenced  in 
the  two  types  of  clinics  are  presented  in  Table  8 for  each  of  the  cate- 
gories of  illness  addressed  in  the  study.  AMIC  return  rates  were  found 
to  be  higher  than  those  manifested  by  a GOC  for  three  of  the  categories 
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Table  8 


PATIENT  RETURN  RATES  BY  CATEGORY  OF  ILLNESS  FOR  GOCs  AND  AMICs 

K 


TYPE  OF  CLINIC 

GOC 

AMIC 

STATISTICAL 

CATEGORY  OF  ILLNESS 

TOTAL  NO.  ’ 
PATIENTS 

PERCENT 

RETURN 

TOTAL  NO. 

PATIENTS 

PERCENT 

RETURN 

SIGNIFICANCE 
( P £ )* 

Eye  (1) 

13 

7.7 

21 

14.3 

.61 

Respiratory  (2) 

129 

13.2 

450 

12.2 

.49 

Gastrointestinal 

(3) 

47 

14.9 

64 

10.9 

.26 

Genitourinary  (4) 

29 

13.8 

104 

9.6 

.29 

Musculo-Skeletal , 
Spine  (5) 

30 

13.3 

7 

0.0 

.02 

Musculo-Skeletal , 
Extremity  (6) 

58 

17.2 

84 

8.3 

.04 

Skin-Regional  (7) 

19 

5.3 

88 

6.8 

.70 

Skin-General  (8) 

42 

9.5 

87 

11.5 

.60 

All  (1-8) 

367 

13.1 

905 

11.0 

.0950 

* 2-tailed  test 
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(Eye,  Skin-Regional,  and  Skin-General);  however,  none  of  these  differences 
were  statistically  significant.  AMIC  return  rates  were  lower  than  GOC 
return  rates  for  all  other  categories.  For  all  but  the  two  musculo- 
skeletal categories,  however,  these  differences  were  not  statistically 
significant.  (At  Appendix  A,  Tables  A1  through  A3,  the  return  rate  data 
are  presented  in  detail  for  each  diagnosis  encountered  within  each  of 
the  algorithms.)  Overall,  for  all  patients  categorizable  among  the  eight 
categories,  the  2.1%  difference  in  patient  return  rate  evidenced  in  favor 
of  the  AMIC  (GOC  = 13.1%,  AMIC  = 11.0%,  p = .0950)  were  not  found  to  be 
statistically  significant  at  the  p _<  .05  level  of  confidence  commonly 
employed.  As  regards  this  "overall"  analysis,  it  is  noted  that  the  fre- 
quency with  which  the  cases  were  observed  among  the  various  categories 
of  illness  were  not  comparable  for  the  two  types  of  clinics  (chi-square 
= 53.1  significant  at  p < .001).  Therefore,  since  the  nonsignificance 

observed  in  the  overall  results  may  be  due  to  the  existence  of  compensa- 
tory differences  within  the  data,  additional  analyses  were  undertaken 
using  artifically  constructed  samples  based  upon  extrapolation  from 
observed  return  rates  and  distributions  to  determine  if  statistically 
significant  overall  findings  would  result  if  the  AMIC-sample  were  treated 
in  GOCs  and  vice-versa. 

4.3.3  Supplemental  Analysis:  AMIC-Equated  Comparison.  The  arti- 
fical  sample  is  shown  in  Table  9.  It  was  created  for  the  GOC  using  the 
sample  distribution  characteristics  (among  the  categories  of  illness) 
of  the  AMIC  while  retaining  the  return  rates  appropriate  to  it's  own 
(the  GOC's)  original  data  for  each  of  the  categories  of  illness.  This 
comparison  addresses  the  question  of  whether  or  not  (given  the  return 
rates  observed  in  the  raw  data)  the  overall  difference  between  the  two 
clinics  would  be  a significant  one  if  the  distribution  of  GOC  patients 
among  the  categories  of  illness  had  been  the  same  at.  that  evidenced 
among  AMIC  patients.  As  observed  in  the  last  column,  the  magnitude  of 
the  overall  difference  is  decreased  (as  is  the  likelihood  that  the 
difference  is  a statistically  reliable  one),  but  the  difference  continues 
to  favor  the  AMIC. 

4.3.4  Supplemental  Analysis:  GOC-Equated  Comparison.  The  re- 
maining possibility  is  that  the  lack  of  a statistical  significance 
evidenced  in  the  original  overall  comparison  (i.e.,  the  using  of  the 
raw  data,  Table  8)  is  due  to  compensatory  characteristics  observed  among 
the  distribution  of  cases  in  the  AMIC  data  (again  as  above,  allowing  the 
return  rate  for  each  category  of  illness  to  remain  the  same  for  the 
respective  clinics).  The  alternate  hypothesis  is,  therefore,  that  if 
the  distribution  of  patients  within  the  AMIC  sample  is  artifically  made 
identical  to  that  of  the  GOC  sample,  then  a significant  difference  would 
result.  In  order  to  test  this  "GOC-equated"  hypothesis,  as  rigorously 
(or  as  "sensitively")  as  the  preceeding  AMIC-equated  hypothesis,  it  was 
first  necessary  to  enlarge  the  GOC  population  to  that  of  the  AMIC. 

(This  was  done  without  disturbing  the  relative  distribution  characteris- 
tics of  the  GOC  data  by  simply  multiplying  the  number  of  patients  within 
each  category  by  the  same  constant.)  Once  this  was  done,  then  the  number 
of  patients  in  each  category  of  illness  for  the  AMIC  data  was  then  equated 
to  the  number  of  patients  evidenced  in  the  GOC  data.  (See  Table  10). 


Table  9 

GOC  VERSUS  AMIC  PATIENT  OVERALL  RETURN  RATE  COMPARISON  UTILIZING  AN 
EXPANDED,  AMIC-EQUATED  REDISTRIBUTION  OF  GOC  PATIENTS 


CATEGORY  OF  ILLNESS  I PATIENTS 


Eye  (1) 

Respiratory  (2) 

Gastrointestinal 

(3) 

Genitourinary  (4) 

Musculo-Skeletal , 
Spine  (5) 

Musculo-Skeletal , 
Extremity  (6) 

Skin-Regional  (7) 

Skin-General  (8) 

All  (1-8) 


t CLINIC 

' AMIC 

TOTAL  NO. 
PATIENTS 

PERCENT 

RETURN 

21 

14.3 

450 

12.2 

64 

10.9 

104 

9.6 

7 

0.0 

84 

8.3 

88 

6.8 

87 

11.5 

905 

11.0 

STATISTICAL 


( P £ )* 


When  the  overall  return  rates  are  compared  under  these  conditions  it  is 
evident  that  the  lower  AMIC  return  rate  which  results  (9.9%)  is  signifi- 
cantly different,  p = .0136,  from  the  GOC  return  rate  (13.1%).  This 
analysis  suggests,  therefore,  that  were  the  proportion  of  AMIC  patients 
observed  in  each  category  of  illness  the  same  as  those  encountered  in  a 
GOC,  the  AMIC's  overall  patient  return  rate  would  have  been  significantly 
lower  than  that  evidenced  by  the  GOCs. 

4.3.5  Additional  Considerations. 

4.3.5. 1 In  summary,  the  above  analyses  suggest  that  if  GOCs  were  to 
treat  patient  populations  distributed  among  the  eight  illness  categories  in 
the  same  manner  as  they  were  encountered  in  the  AMICs,  there  would  be  no 
statistically  significant  difference  in  their  overall  patient  return  rates; 
however,  if  the  reverse  situation  were  to  exist,  the  patient  return  rate  for 
the  AMIC  would  be  significantly  lower  than  that  of  the  GOC. 

4. 3. 5. 2 While  the  above-cited  findings  regarding  patient  return  rates 
are  considered  to  be  complimentary  to  the  AMOSIST  Program  (i.e.,  the  patient 
return  rate  for  patients  treated  in  a physician  extender  program  is  no 
higher — and,  perhaps,  somewhat  lower — than  for  patients  treated  by  physi- 
cians), there  exists  one  significant  caution  to  be  cited:  The  reason  for 
this  condition  may  be  that  the  GOC-treated  patient  population  suffers 
illnesses  which  are  relatively  more  severe  or  complicated  than  those  treated 
in  an  AMIC.  It  is,  in  fact,  the  purpose  of  the  triage  function  of  the 
AMOSIST  Program  to  steer  away  from  the  AMIC  all  patients  who  are  viewed  as 
having  something  more  than  an  acute  minor  illness.  Furthermore,  many  (if 
not  most)  of  the  illnesses  addressed  by  the  AMOSIST  Program  are  viewed  as 
self-limiting  conditions  for  which  the  treatment  is  intended  to  effect 
little  more  than  symptomatic  relief.  Since  there  exists  no  data  with  which 
to  further  examine  the  issue,  the  above-cited  possibility  remains  as  merexy 
plausible  conjecture.  However,  if  one  makes  the  assumption  that  each  type 
of  care  provider  is  treating  patients  whose  severity  and  complexity  of 
illnesses  are  appropriate  to  the  level  of  knowledge  and  training  of  their 
respective  care  providers,  then  the  direct  comparison  (and  the  results 
thereof)  is  given  some  legitimacy. 

4.3.5. 3 A second  factor  to  be  considered  in  evaluating  the  return-rate 
findings  is  the  previously  mentioned  bias  in  the  manner  in  which  the  AMICs 
appearing  in  this  portion  of  the  study  were  chosen.  That  process  was  in- 
tended to  assure  that  those  included  were  among  the  most  efficiently  and 
effectively  run  programs  of  those  which  were  operating  in  most  accord  with 
the  AMOSIST  Program  guidelines.  Hence,  the  present  findings  are  likely  to 
be  more  representative  of  the  "potential"  existing  within  the  program  than 
they  are  of  the  "typical"  or  average  AMIC. 

4.3.6  AMOSIST  Consultation  versus  Non-Consultation  Return  Rates. 

In  addition  to  the  GOC  versus  AMIC  patient  return  rate  comparisons, 
the  data  were  further  examined  to  determine  if  physician  consultations 
differentially  affected  the  patient  return  rates.  This  was  done  by  com- 
paring the  return  rates  evidenced  among  patients  initially  treated  by 
AMOSISTs  who  consulted  with  a physician  with  those  initially  treated  by  an 
AMOSIST  who  did  not  consult  with  a physician.  These  data  are  depicted  in 
Table  11.  The  findings  show  that  although  some  statistically  significant 
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Table  11 


NON-DIRECTED  PATIENT  RETURN  RATES  FOR  AMOSIST-TREATED  PATIENTS 
WITH  AND  WITHOUT  PHYSICIAN  CONSULTATION 


NON-DIRECTED  RETURN  VISITS 

WITH  PHYSICIAN 
CONSULT  ON  INITIAL  VISIT 

WITHOUT  PHYSICIAN 
CONSULT  ON  INITIAL  VISIT 

ALGORITHM 

TOTAL  NO. 

OF  CASES 

NUMBER 

OF  CASES 

PERCENT 

RETURNED 

NUMBER 

OF  CASES 

PERCENT 

RETURNED 

Eye  (1) 

21 

14 

14.3 

7 

7.4 

Respiratory  (2) 

450 

235 

13.2 

215 

10.8 

Gastrointestinal 

(3) 

64 

44 

4.5 

20 

25.0 

Genitourinary  (4) 

104 

69 

13.0 

35 

2.9 

Musculo-Skeletal , 
Spine  (5) 

7 

6 

0.0 

1 

0.0 

Musculo-Skeletal , 
Extremity  (6) 

84 

63 

6.4 

21 

9.5 

Skin,  Regional 
(7) 

88 

73 

8.2 

15 

0.0 

Skin,  General  (8) 

87 

56 

10.7 

31 

12.8 

All  (1-8) 

905 

560 

10.7 

345 

■ 

10.8 

differences  exist  between  these  two  groups  among  the  various  algorithms, 
the  results  of  the  comparisons  do  not  consistently  favor  either  consulta- 
tion or  non-consultation  patients.  In  fact,  as  shown  at  the  bottom  line 
of  the  table,  the  overall  results  for  these  two  groups  are  nearly  identi- 
cal with  non-directed  patient  return  rates  of  10.7  percent  and  10.8  percent 
for  the  consultation  and  non-consultation  groups , respectively . 

4.4  Additional  Issues  and  Considerations. 

4.4.1  Evaluation  of  Overlay  Auditing  Procedure.  In  the  first  of  the 
series  of  reports  concerning  this  study  (Schopper,  1978a),  it  was  indicated 
that  there  did  not  appear  to  exist  an  efficient  means  of  meeting  the  auditing 
requirements  stated  in  the  AMOSIST  Program's  guidelines.  Whereas  the  algo- 
rithm development  and  validation  studies  ongoing  in  the  AMIC  at  Brooke  Army 
Medical  Center  (BAMC)  are  supported  by  a computerized  audit  system,  the 

AMIC  sites  in  all  other  locations  must  rely  on  manual  DCS  auditing  procedures 
which  depend  totally  upon  the  expertise  residing  in,  and  the  amount  of  time 
available  to,  the  local  AMOSIST  Physician  and/or  the  AMIC  NCOIC.  The  criti- 
cism made  was  that  it  was  unlikely  (given  the  algorithms,  DCSs,  and  data 
collection  procedures  employed)  that  there  existed  sufficient  time  in  many 
of  the  AMICs  to  manually  perform  audits  of  DCSs  in  the  amount  required  and 
in  the  detail  necessary  to  make  the  results  both  representative  and  mean- 
ingful to  either  the  auditor  or  the  AMOSIST  (assuming  that  the  latter  is 
given  feedback  concerning  the  quality  of  his  work) . As  a result  of  these 
findings  and  criticism,  it  was  suggested  that  one  possible  alternative  to 
the  manual  auditing  procedures  might  be  the  use  of  an  overlay  transparency 
approach  such  as  was  used  in  evaluating  the  DCSs  analyzed  in  the  present 
report.  Based  upon  the  experiences  of  the  writer,  the  overall  evaluation 
of  this  approach  is  that  it  is  too  limited  in  its  applications  to  justify 
a recommendation  for  widespread  use  within  the  present  system  of  operation. 
While  a rigorous  audit  was  possible  in  the  present  study,  such  a system 
(a)  is  vulnerable  to  mismatch  as  a result  of  distortions  in  the  DCSs  arising 
from  differences  among  the  various  types  and  models  of  local  photocopiers 
employed,  and  more  seriously  (b)  is  capable  of  addressing  only  about  20 
to  25  percent  of  the  records.  (A  more  complete  discussion  of  this  audit 
technique  appears  in  Appendix  E.) 

4.4.2  Alternative  Integrated  Data  Collection  Process. 

4.4.2. 1 Background.  Previous  paragraphs  have  summarized  and  discussed 
the  findings  pertaining  to  the  extent  to  which  AMOSISTs  recorded  their 
findings,  assessment,  and  treatment  plan  in  accordance  with  the  algorithms 
provided  for  their  use.  These  results  were  not  complimentary  to  the  AMOSIST 
Program.  When  contrasted  with  the  findings  from  the  AMIC  at  BAMC,  the 
results  clearly  indicate  the  need  for  intervention  and  improvement.  Although 
several  factors  were  cited  as  being  possible  contributors  to  the  poor  per- 
formance of  the  AMOSISTs  evaluated  in  the  present  study  (vis-a-vis  that  of 
BAMC  AMOSISTs)  it  is  believed  that  another  factor  contributes  to  the  errors 
observed;  that  being  the  fact  that  the  data  collection  procedures  employed  re- 
flect data  collection  processes  which  do  not  require  the  AMOSIST  to  provide 


physical  evidence  that  he  has  or  has  not  utilized  the  algorithm  in  ar- 
riving at  his  diagnosis  and  formulating  his  treatment  plan.  For  AMOSISTs 
serving  at  facilities  other  than  BAMC , the  algorithms  are  contained  in  a 
rather  large,  separate  manual;  a manual  which  is,  in  actuality,  rarely 
consulted  (and  is  even  occasionally  absent  from  the  premises,  ^chopper, 

1978a).  For  these  AMOSISTs,  there  exists  a need  to  integrate  the  algorithm 
and  the  DCS  into  a single  document,  a revised  DCS  which  portrays  the  al- 
gorithm itself  and  demands  that  the  AMOSIST  record  his  findings  in  a 
manner  which  clearly  documents  the  logic  he  has  employed  in  examining  and 
treating  his  patient.  For  AMOSISTs  serving  at  BAMC,  this  need  has  been 
partially  met.  The  DCSs  for  the  algorithms  which  have  been  validated  or 
are  being  studied  are  multipart,  pressure  sensitive,  carbonized  forms 
which  contain  the  history  and  examination  findings  on  one  side  (Figure  2) , 
of  each  page  and  the  algorithm  itself  on  the  reverse  side  of  the  last  page 
(Figure  3).  The  BAMC  AMOSISTs  do  not  have  the  need  to  consult  a separate 
manual  to  avail  themselves  to  the  logic  of  the  algorithm;  they  merely 
have  to  turn  the  DCS  over.  Hence,  this  system  of  data  recording  is  not 
only  considerably  more  convenient,  but  is  also  a much  more  psychologi- 
cally desirable  one  since  the  need  (or  requirement)  to  consult  the 
algorithm  is  much  less  obvious  to  the  patient  when  the  referent  is  an 
integral  part  of  the  AMOSISTs  work  sheet.* 

4. 4. 2. 2 Existing  Two-Step  Process.  As  cited  previously,  however,  the 
BAMC  DCSs  only  partially  meet  the  requirements  stated;  they  do  not  meet  the 
need  to  have  readily  obvious  documentation  of  the  logic  employed  by  the  AMOSISTs 
in  determining  the  diagnosis  and  treatment  plan.  Since  the  need  relates  as  much 
(if  not  more)  to  the  audit-related  aspects  of  the  program  as  it  does  to 
the  data  recording  aspects,  the  fact  that  this  capability  does  not  ex- 
ist in  BAMC's  DCS  is  not  nearly  so  critical  as  it  is  for  other  facilities  be- 
cause the  data  from  the  BAMC  DCSs  are  transferred  to  computer  cards  and 
the  (very  thorough)  audit  is  performed  by  a computer.  At  other  AMICs , 
however,  computerized  audits  are  not  possible;  consequently,  they  must 
rely  totally  on  manual  audits  unaided  by  any  mechanical  device  (e.g., 
overlays).  To  provide  to  these  AMICs  a DCS  which  clearly  depicts  the 


*As  pointed  out  in  the  initial  report,  one  reason  why  non-BAMC  AMOSISTs 
do  not  consult  the  AMOSIST  Manual  is  the  negative  impression  they  feel  is 
conveyed  to  the  patient  regarding  their  competence  by  the  need  to  physi- 
cally thumb  through  a book  as  they  complete  the  patient's  examination 
and  develop  a treatment  plan.  Hence,  the  depiction  of  the  algorithm  on 
the  DCS  is  apt  to  increase  the  use  of  the  algorithms  not  only  because  it 
is  more  readily  accessible,  but  also  because  it  may  lessen  the  AMOSIST 's 
concern  that  the  patient  would  believe  him  to  be  inept  because  of  his 
use  of  same . 


36 


logic  used  by  the  AMOSIST  in  examining  and  treating  his  patients  would,  1 

it  is  believed,  greatly  ease  the  auditing  task  and,  therefore,  increase  1 

the  likelihood  that  meaningful  audits  could  be  performed  on  a daily  basis. 

4.4, 2. 3 Proposed  Integrated  Process.  An  example  of  the  type  of  algorithm 
which  is  proposed  is  shown  in  Figure  5.  The  algorithm  shown  is  the  BAMC  LR1  alg° 
rithm  and  the  suggested  modification  consists  merely  of  inserting  a directional, 
blank  check  point  box  in  the  lines  interconnecting  the  various  logic  t 

boxes.  The  Instructions  to  the  AMOSISTs  would  be  for  them  to  indicate  3 

their  logic  flow  by  placing  a mark  (e.g.,  "/")  in  the  box  appropriate  j 

to  their  decision  following  each  decision/ instructional  block  through 
which  they  passed  during  the  course  of  their  examination.  If  this  system 
were  adopted  in  conjunction  with  instructions  to  the  AMOSIST  to  indicate 
at  what  specific  point  a logic  overrule  was  imposed  by  a physician  as 
the  result  of  a consultation,  e.g.,  a strike  through  the  checkmark  ("/") 
as  is  employed  in  the  BAMC  data  recording  system,  then  (theoretically) 
all  DCSs,  regardless  of  the  number  of  diagnoses,  could  be  readily  audited 
by  hand.  Since  the  logic  trail  would  be  unambigiously  depicted  on  the 
reverse  side  of  the  last  page  of  each  DCS  (the  front  side  would  retain 
much  of  the  same  information  contained  in  the  present  DCSs),  the  level 
of  skill  and  medical  knowledge  required  to  perform  the  audit  would  be 
markedly  reduced.  Hence,  the  existing  HSC  guidance  that  physicians  per- 
formed all  DCS  audits  could  be  substantially  relaxed  (e.g.,  require  that 

he  perform  an  audit  of  only  some  randomly  selected  small  percent  of  those  3 

previously  audited  by  the  NCOIC) . 

4. 4. 2. 4 Reaction  to  Proposal.  Albeit  there  exist  no  data  with 
which  to  assert  the  superiority  of  the  proposed  integrated  data  col- 
lection scheme,  the  writer  has  had  the  opportunity  to  discuss  the  con- 
cept with  the  Assistant  Chief,  Department  of  Medicine,  BAMC,  LTC  Wolcott. 

(LTC  Wolcott  is  also  the  physician  who  was  instrumental  in  developing  the 
previously  cited  clinical  validation  study  which  has  been  ongoing  in  the 
AMIC  at  BAMC  for  the  past  three  years,  see  Berger  and  Wolcott,  1967; 

Wolcott,  Berger,  Tompkins,  and  Wood,  1976;  Wolcott,  Tompkins,  Wood,  Sox, 
and  Berger,  1976).  Dr.  Wolcott  enthusiastically  endorsed  the  concept  and 
the  proposed  modification  to  the  DCSs.  Additionally,  as  a result  of  sub- 
sequent communication  with  the  AMOSIST  Branch,  AHS,  that  organization  has 
informally  indicated  that  it  intends  to  field  a revision  of  their  DCSs 
which  will  incorporate  the  proposed  integrated  data  collection  procedure. 

4.4.3  Medico-Legal  Considerations. 

4.4.3. 1 Background.  In  the  initial  report  of  the  present  series, 
one  substantial  consideration  addressed  was  the  medico-legal  issue  as  it 
pertained  to  the  AMOSIST  program.  Among  the  specific  concerns  cited  was 
the  perceived  vulnerability-to-legal-suit  arising  from  alleged  improper 
care  having  been  provided  by  an  AMOSIST;  a vulnerability  which  was  viewed 
as  being  particularly  acute  in  view  of  the  relatively  inadequate  proce- 
dures employed  by  most  sites  to  monitor  and  audit  the  extent  to  which 
AMOSISTs  adhere  to  the  algorithms.  Albeit  the  present  findings  are  in 
consonance  with  the  observations  intially  reported,  such  does  not  assert 
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that  the  AMOSIST  Program  has,  in  fact,  been  a legal  liability  to  the  Army 
Medical  Department  (AMEDD) . 

4. 4. 3. 2 Medical  Claims.  The  most  appropriate  data  with  which  to 
assess  the  extent  to  which  the  AMOSIST  Program  has  been  a medico-legal 
liability  are  those  depicting  the  actual  incidence  with  which  malpractice 
claims  have  been  filed  against  the  US  Government  involving  AMOSISTs.  While 
data  on  this  specific  issue  are  in  the  process  of  being  collected,  informal 
communication  with  the  Department  of  Legal  Medicine,  Armed  Forces  Institute 
of  Pathology  AFIP) , indicates  that  the  results  of  these  efforts  will  likely 
show  that  very  few,  if  any,  such  claims  have  been  filed.  (The  actual  find- 
ings of  this  inquiry  will  be  analyzed  and  reported  upon  in  the  brief  execu- 
tive summary  which  is  to  appear  following  the  final  report  in  the  present 
series.)  Hence,  on  the  basis  of  informal  estimates,  it  appears  that  the 
AMOSIST  Program  has  not,  to  date,  proven  itself  to  have  been  a medico- 
legal liability  to  the  AMEDD. 

4.4. 3. 3 Moderating  Factor;  Severity-of-Illness . Should  the  actual 
findings  of  the  AFIP  search  prove  to  be  as  favorable  as  envisioned,  the 
interpretation  should  nonetheless  reflect  the  consideration  voiced  earlier 
regarding  the  fact  that  the  illness  treated  by  AMOSISTs  should  be,  for 
the  most  part,  illnesses  which  are  less  severe  than  those  treated  by 

more  highly  trained  care  providers.  Therefore,  it  might  be  argued  that  the  errors 
made  by  AMOSISTs  are  less  likely  to  have  a serious,  enduring  impact  upon 
the  recipient  of  the  care  than  are  errors  or  mistakes  made  by  other  care 
providers  who  may  treat  more  serious  illnesses.  While  there  may  exist 
some  merit  in  such  an  argument,  it  is  pointed  out  that  claims  which  are 
filed  involving  an  AMIC  are  likely  to  be  those  which  occur  as  a result  of 
an  error  which  results  in  a significant  illness.  However,  given  the  error 
rates  cited  herein,  much  would  seem  to  depend  upon  the  effectiveness  of 
the  triage  system  in  selecting  only  patients  with  acute  minor  illnesses 
to  be  seen  in  the  AMIC.  If  the  selection  process  was  poor,  then  the  like- 
lihood that  an  error  would  eventuate  into  a more  serious  illness  would 
appear  to  be  considerably  higher  than  if  the  selection  process  was  good. 

However,  the  findings  of  the  initial  report  of  this  series  indicated  that 
the  triage  process  was,  if  anything,  less  well  monitored  and  audited  than 
was  the  treatment  DCS  function.  Hence,  the  temptation  to  reintroduce  the 
severity-of-illness  issue  to  explain  the  findings  does  not  result  in  as 
powerful  a mitigating  factor  as  might  first  be  thought. 

4.4. 3.4  Defensibility . While  the  severity-of-illness  issue  is  one 
which  beclouds  several  of  the  comparisons  made  in  the  present  study,  its 
relevance  to  the  medico- legal  aspect  of  the  AMOSIST  program  differs  ac- 
cording to  the  vantage  point  taken.  As  cited  above,  it  is  anticipated 
that  the  actual  assessment  of  the  extent  to  which  claims  have,  in  fact, 
eventuated  from  the  care  which  has  been  provided  at  AMICs  will  be  very 
small.  This  is  a different  question,  however,  from  that  of  the  legal 
defensibility  of  a care  provider's  actions  once  a claim  has  been  initiated. 

Both  of  these  factors,  actual  incidence  of  claims  filed  and  the  defensi- 
bility of  same,  pertain  to  the  superordinate  issue  of  "vulnerability- 
to-successful-suit . " The  present  study  findings  of  non-significant 
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differences  in  patient  return  rates  between  GOCs  and  AMICs,  in  conjunction 
with  the  anticipated  finding  that  very  few,  if  any,  claims  have  been  made 
against  the  government  which  implicate  AMIC  personnel,  suggest  that  the 
AMOSIST  Program  has  fared  well  as  regards  the  frequency-related  component 
of  the  "vulnerabili ty-to-successf ul-suit"  issue.  Nevertheless,  the  findings 
of  the  present  report  which  reflect  the  AMOSISTs  high  DCS  error  rates,  and 
the  findings  of  the  initial  report  which  reflect  inadequate  auditing  pro- 
cedures, suggest  that  the  program  is  definitely  "at  (high)  risk"  as  re- 
gards the  defensibility-against-suit"  component  of  the  issue. 

4. 4. 3. 5 Acceptability  of  Treatment  Procedures.  Another  factor  which 
may  be  relevant  to  the  medico-legal  issue  (albeit  it  is  one  which  is  independent 
t of  the  results  of  the  present  study)  is  that  of  the  validity  of  the  algorithms  pre- 

sently in  use  outside  of  BAMC.  Even  if  the  present  findings  regarding  DCS  error 
rate  had  been  equal  to  that  evidenced  by  BAMC  AMOSISTs,  the  procedures  themselves 
would  seem  to  be  open  to  legal  question  since  they  have  not  yet  undergone  any 
formal  clinical  evaluation  to  attest  to  their  efficacy.  While  it  may  be 
unlikely  that  the  question  would  be  asked,  or  if  asked,  that  the  algorithms 
would  be  found  wanting,  cognizance  must  be  given  to  the  fact  that  the 
legal  acceptability  of  a procedure  is  predicated  for  the  most  part  upon 
its  acceptance  in  the  medical  community.  An  acceptance  of  a procedure  in 
the  medical  community  is  frequently  judged,  at  least  in  some  substantial 
part,  by  its  appearance  in  the  professional  literature.  On  this  basis, 
the  algorithms  appearing  in  the  AMOSIST  Manual  may  be  "at  risk"  since 
this  writer  has  no  knowledge  of  any  formal  evaluation  of  them  that  has 
appeared  in  the  professional  literature.  This  vulnerability  does  not 
apply  to  BAMC-validated  algorithms,  since  the  results  of  their  evaluations 
I have  been  published;  e.g.,  Tompkins  et  al  (1977). 

5.  SUMMARY  OF  FINDINGS 

5. 1 Findings  Related  to  Safety  of  Care  Delivered. 

5.1.1  Presence  of  Vital  Signs  on  DCS.  The  vital  signs  were  found 
to  be  relatively  infrequently  recorded.  Among  the  sample  of  282  single 
diagnosis  DCSs  completed  solely  by  AMOSISTs,  only  60.9  percent  had  the 
temperature  recorded,  none  evidenced  a pulse  rate,  0.7  percent  depicted 
the  respiratory  rate,  and  12.6  percent  showed  the  patient's  blood  pres- 
sure. 

5.1.2  Treatment  DCS  Error  Rates.  Two  hundred  eighty  two  AMOSIST 
completed  DCSs  showing  one  diagnosis  and  no  evidence  of  a physician  con- 
sultation were  examined  to  determine  to  what  extent  the  data  recorded 
therein  was  in  accord  with  the  logic  and  direction  provided  in  the  algo- 
rithms. For  the  purpose  of  the  study,  two  independent  types  of  errors 
were  identified  which  correspond  to  differences  reflected  on  the  data 
entry  points  of  the  DCSs  themselves.  Type  A errors  were  those  denoting 
the  presence  of  a positive  (+)  entry  in  data  entry  points  which  had  been 
specially  marked  on  the  DCS  (with  an  asterisk)  to  inform  the  AMOSIST  that 
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if  the  symptom  corresponding  to  same  existed,  he  must  seek  a physician 
consultation  prior  to  completing  a treatment  plan  for  a patient.  Type  B 
errors  were  those  which  denoted  the  presence  of  entries  to  the  remaining 
(non-asterisked)  data  entry  points  which  were  contrary  to  the  logic  of 
the  algorithm.  These  logic-related  errors  should  also  have  resulted  in  a 
consultation  with  a physician  since  an  AMOSIST  is  not  permitted  to  assume 
the  entire  responsibility  for  the  care  of  the  patient  unless  that  care  -is 
in  total  accord  with  the  algorithm.  Both  types  of  error,  therefore,  re- 
flect errors  of  omission  since  both  should  have  resulted  in  a consulta- 
tion with  a physician — an  action  which  did  not  occur  in  the  sample  of 
DCSs  evaluated.  Independently  considered,  the  results  of  the  analysis 
for  Type  A errors  indicated  that  a physician's  consultation  was  not 
rendered  as  required  for  73  percent  of  the  282  cases  evaluated.  A 
separate  analysis  addressing  only  Type  B errors  showed  that  an  even  greater 
proportion,  90.1  percent,  of  the  DCS  should  have  reflected  a physician's 
involvement  because  of  some  discrepancy  between  the  algorithm's  logic 
and  the  AMOSIST' s recordings  regarding  either  the  symptoms  indicated  or 
the  treatment  plan  developed.  Considered  conjointly,  the  findings  were 
that  nearly  all  of  these  282  DCSs,  97.1  percent,  evidenced  an  error  of 
omission — the  lack  of  a consultation  with  a physician. 

5 . 2 Findings  Related  to  Medical  Effectiveness  of  Care  Delivered 
(Patient  Return  Rates) . Among  the  eight  general  categories  of  illness 
addressed  by  the  AMOSISTs  algorithms,  the  overall  proportion  of  AMIC- 
treated  patients  who  made  non-directed  returns  to  obtain  additional  care 
for  the  same  or  a related  illness  as  had  been  treated  within  the  clinic 
within  the  previous  14  days  was  not  significantly  different  than  that 
evidenced  among  GOC-treated  patients. 

5 . 3 Additional  Findings. 

5.3.1  DCS  Utilization.  Overall,  among  all  905  DCSs  of  AMIC-treated 
patients  included  in  the  retrospective  record  audit,  the  data  were  entered 
upon  current  DCSs  for  83.6  percent  of  the  patients,  upon  outdated  DCSs  for 
3.2  percent  of  the  patients,  and  upon  SF  600s  for  the  remaining  13.2  per- 
cent of  the  patients. 

5.3.2  Frequency  of  Occurrence  Among  Diagnoses  Rendered.  Among  all 
905  DCSs  included  in  the  retrospective  audit,  nearly  40  percent  (20 
diagnoses)  of  the  52  specific  diagnoses  included  in  the  present  AMOSIST 
Manual  occurred  two  times  or  less;  nine  of  these  did  not  occur  at  all. 

5.3.3  DCS  Format.  The  format  of  the  DCSs  presently  in  use  in  the 
AMOSIST  Program  does  not  provide  a clear  and  readily  understandable  record 
of  the  logic  employed  by  AMOSIST  in  arriving  at  a diagnosis  and  deter- 
mining a treatment  plan.  Nor  do  the  data  recording  procedures  employed 
enable  AMOSIST  physicians  performing  requested  consultations  to  document 
the  extent  and  nature  of  their  involvement  in  the  treatment  of  an  aMOSIST's 
patient . 

6.  CONCLUSIONS. 
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6. 1 Safety  of  Care 

6.1.1  As  a result  of  the  exceptionally  high  error  rate,  the  AMOSIST 
program  is  viewed  to  be  "at  risk"  from  a medico-legal  perspective. 

6.1.2  The  present  AMOSIST  Manual  is  overburdened  with  instruction  and 
logic  pertaining  to  infrequently  utilized  diagnoses;  a condition  which  is 
believed  to  be  contributing  to  the  lack  of  its  use  by  AMOSISTs  in  the  field, 
and  consequently,  to  the  high  error  rate  evidenced. 

6.1.3  The  validity  and  efficiency  of  existing  and  envisioned  treat- 
ment DCS  auditing  procedures  is  degraded  by  the  lack  of  use  of  current, 
program-prescribed  treatment  DCSs  to  record  the  findings,  treatment,  and 
disposition  of  all  AMOSIST-treated  patients. 

6.1.4  The  format  of  the  current  data  collection  sheets  and  the  pro- 
cedures used  by  the  physicians  to  record  the  extent  and  nature  of  their 
impact  upon  the  treatment  of  an  AMOSIST's  patient  are  incompatible  with  a 
requirement  to  efficiently  and  effectively  perform  manual  audits  of 
treatment  DCSs. 

6.2  Effectiveness  of  Care.  The  AMOSIST  Program  has  the  potential  to 
deliver  care  which  is  not  significantly  different  in  its  effectiveness  than 
that  which  would  be  delivered  by  physicians  treating  a comparable  sample  of 
patients. 

7 . RECOMMENDATIONS . 

7 . 1 Principal  Recommendations . 

The  principal  recommendations  arising  from  the  present  report  are 

that: 

a.  New  series  of  DCSs  and  data  collection  procedures  be  developed 
that  will  clearly  depict: 

(1)  the  logic  utilized  by  the  AMOSIST  in  examining  and  treating 
their  patients,  and 

(2)  the  nature  and  extent  of  physician  impact  upon  the  algorithmic 
care  delivery  process  which  results  from  AMOSIST-requested  physician  consul- 
tations. 

b.  HSC  develop  binding,  enforceable  guidance  for  users  of  the 
AMOSIST  Program  that  requires  daily  formal  audits  to  be  locally  performed 
and  documented  and  that  the  results  of  same  (as  applies  to  each  AMOSIST) 
be  communicated  to  each  AMOSIST  on  a daily  basis. 

c.  HSC  institute  a command  quality  control  program  consisting  of 
two  basic  elements. 
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(1)  The  MEDDAC  report  results  of  their  AMOSIST  audit  program  on 
a routine  basis. 

(2)  A requirement  that  each  program  receive  an  on  site  audit  of 
randomly  selected  records  by  HSC  headquarters  personnel  as  part  of  routine 
staff  visits  and/or  the  IG  visit.  This  should  be  done  as  frequently  as 
practical  but  as  a minimum  annually. 

7.2  Additional  Recommendations.  The  following  actions  are  recommended 
for  implementation  at  the  appropriate  time  as  determined  by  the  DCSPA,  HSC, 
as  part  of  the  AMOSIST  Program  improvement: 

a.  That,  to  the  extent  possible,  the  logic  employed  in  each  such 
DCS  series  be  that  derived  from  or  to  be  added  in  the  research  efforts  on- 
going at  the  BAMC  AMIC. 

b.  That  the  algorithms  and  DCSs  first  revised  be  those  already 
validated  by  the  BAMC  University  of  Washington  (BAMC/UW)  Study. 

c.  That  a hierarchy  of  priorities  be  developed  for  the  remainder 
of  the  algorithms  to  be  addressed,  and  that  this  hierarchy  be  consistent  to 
the  extent  possible  with  the  sequence  in  which  BAMC/UW  intends  to  address 
similar  algorithms. 

d.  That  any  revision  of  the  AMOSIST  Manual  and  associated  DCSs 
reflect  the  results  of  an  evaluation  undertaken  to  determine  the  incidence 
of  the  diagnoses  included  in  order  to  increase: 

(1)  the  efficiency  and  utility  of  the  program,  and 

(2)  the  likelihood  that  the  products  of  the  revision  are,  in 
fact,  employed  by  the  intended  users. 

8 . REFERENCES . 

Burger,  L.,  Wolcott,  B.  W. , Physician  extenders  in  the  delivery  of 
non-appointed  ambulatory  medical  care  in  the  military.  Military  Medicine, 
1977,  141,  357-359. 

Schopper,  A.  W.,  Health  Care  Delivery  Study  Proposal:  AMOSIST  Program 
Field  Evaluation  (Project  APFE) . (As  Incl  1 to  Ltr,  US  Army  Academy  of 
Health  Sciences,  HSA-CHC,  subject:  Proposed  Final  Study  Protocol  for  the 
AMOSIST  Program  Field  Evaluation,  20  April  1976.) 

Schopper,  A.  W. , AMOSIST  Program  Field  Evaluation  Phase  I:  Operational 
Characteristics  and  Program  Acceptance,  Health  Care  Studies  Division  Report: 
Health  Care  Studies  Division,  Academy  of  Health  Sciences  (HCSD  Report 
No.  78-002-A) , March  1978(a). 

Schopper,  A.  W. , AMOSIST  Program  Field  Evaluation:  Physician  Savings 
and  Cost  Effectiveness ,~  Health  care  Studies  Division,  Academy  of  Health 
Sciences  (HCSD  Report  No.  78-002-B) , August  1978(b). 


43 


Sox,  H.  C.,  Jr.,  Sox,  C.  H. , and  Tompkins,  R.  K.,  The  Training  of 
Physicians'  Assistants:  The  Use  of  a Clinical  Algorithm  System  for  Patient 
Care,  Audit  of  Performance  and  Education.  The  New  England  Journal  of  Medicine, 
1973,  288,  818-824. 

Tompkins,  R.  K. , Wood,  R.  W.,  Wolcott,  B.  W. , Walsh,  B.  T.,  The 
Effectiveness  and  Cost  of  Acute  Respiratory  Illness  Medical  Care  Provided 
by  Physicians  and  Algorithm-Assisted  Physicians'  Assistants.  Medical  Care, 
1977,  T5,  991-1003. 

Wolcott,  B.  W. , Burger,  L.,  Tompkins,  R.  K. , Wood,  R.  W.,  Evaluating 
physicians  extenders  care:  Algorithm-associated  management  of  ambulatory 
patients  with  acute  respiratory  illnesses.  Speech  to  American  Federation 
of  Clinical  Investigation.  Atlantic  City,  New  Jersey,  1976. 

Wolcott,  B.  W. , Tompkins,  R.  K.,  Wood,  R.  W.,  Sox,  H.,  Burger,  L.: 
Algorithm  directed  care  of  upper  respiratory  infections  by  three  classes 
of  provider  process  and  outcome.  Speech  to  American  College  of  Physicians, 

San  Antonio,  Texas,  1976. 

Wolcott,  B.  W. , Algorithm  Directed  Troop  Medical  Care  (ADTMC)  Project, 
Clinical  Investigation  Service,  Brooke  Army  Medical  Center,  1977. 

Ambulatory  Patient  Care  Model  #13,  US  Army  Health  Services  Command, 
HSPA-A,  August  1976. 

Army  Regulation  40-400,  Patient  Administration,  1 October  1978. 

Army  Regulation  40-403,  Health  Records,  15  June  1974  (Interim  change, 
March  1977) . 


44 


APPENDIX  A 

TYPE  A ERRORS  FOR  URI  ILLNESSES 

A. 1 As  indicated  in  the  principal  text  of  the  report  (paragraph 
4. 2. 3. 2,  page  7)  the  majority  of  the  AMOSIST-only , single-diagnosis  DCSs 
pertained  to  URI-related  ailments.  Since  all  URI  diagnoses  appear  upon 
the  same  DCS  (see  Figure  I,  page  13),  it  is  possible  to  address  these 
illnesses  further.  Table  A-l  depicts  the  breakout  of  all  Type  A errors 
encountered  among  the  various  URI  diagnoses  encountered  that  met  the 
AMOSIST-only,  single-diagnoses  criteria.  In  the  left  hand  column  of  that 
table  are  listed  all  possible  Type  A errors.  According  to  the  instruc- 
tion provided  AMOSISTs,  any  entry  in  one  of  those  areas  listed  should 
have  resulted  in  a consultation  with  the  supervising  physician.  The 
vertical  columns  within  the  body  of  the  table  reflect  the  distribution 
of  such  errors  according  to  diagnostic  category.  The  extreme  right-hand 
column  summarizes  the  frequency  of  occurrence  of  such  errors  across  all 
diagnostic  categories. 

A. 2 It  is  apparent,  upon  examining  the  summary  column,  that  the  vast 
majority  of  the  Type  A errors  can  be  accounted  for  by  only  two  specific 
errors:  the  failures  to  consult  with  the  physician  for  instances  wherein 

(a)  patients  were  already  taking  physician-prescribed  medication,  and 

(b)  AMOSISTs  prescribe  medications  which  were  not  cited  in  the  AMOSIST 
Manual.  Such  errors  accounted  for  22.5  percent  and  43.8%,  respectively, 
of  all  Type  A URI  errors.  Only  one  other  specific  error  occurred  in  more 
than  10  percent  of  the  DCSs,  i.e.,  that  wherein  an  AMOSIST  did  not  consult 
the  physician  concerning  an  additional  subjective  patient  symptom  which 

he  recorded. 
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APPENDIX  B 

TYPE  B,  LOGIC-RELATED  ERRORS  FOR  INDIVIDUAL  URI  DIAGNOSES: 

DISCUSSION  AND  TABLES 

B.l  General . Whereas  the  DCS  entries  which  are  relevant  to  Type  A 
URI  errors  are  reasonably  consistent  across  diagnoses,  and  are  amenable, 
therefore,  to  summarization  within  a single  table,  such  is  not  the  case 
with  Type  B URI  errors  since  the  logic  contingencies  which  pertain  to  each 
DCS  entry  vary  from  diagnosis  to  diagnosis.*  As  a result,  the  Type  B 
logic  errors  which  pertain  to  the  URI  diagnoses  must  be  depicted  separately 
for  each  diagnosis.  The  findings  for  the  most  frequently  occurring  diag- 
nosis, non-specific  acute  pharyngitis,  are  shown  in  Table  B-l.  (This  is 
done  for  the  remainder  of  the  respiratory  diagnoses  in  Tables  B-2  through 
B-10.)  Within  the  table,  the  requirements  which  were  not  met  are  cited 
along  with  the  percentage  of  cases  for  which  that  requirement  was  not 
realized  in  the  DCS  entries.  Entries  are  grossly  divided  into  two  cate- 
gories, symptom-related  entries  and  treatment-related  entries. 

B. 2 Symptom-related  Type  B Errors. 

B.2.1  The  symptom- related , Type  B logic  errors  cited  in  Table  B-l 
for  Non-specific  Pharyngitis  example  the  nature  of  some  of  the  types  of 
logic  errors  encountered  among  the  DCSs.  The  first  two  cited,  "absence 
of  positive  throat  culture  within  last  10  days"  and  "absence  of  past 
history  of  rheumatic  fever"  provide  concrete  examples  of  the  second  of 
f the  two  types  of  "errors  of  omission"  discussed  earlier  (re  paragraph 

| 4. 2. 3. 3. 3).  All  13  of  these  errors  were  due  to  the  absence  of  any  entry 

in  the  space  corresponding  to  these  conditions.  Since  they  were  both,  in 
fact,  specifically  addressed  with  the  algorithm,  there  should  have  ex- 
isted a negative  response  entered  on  the  DCS  for  each  item  regarding 
both  of  these  conditions  if,  in  actuality,  the  patient  did  lack  them. 

B.2.2  The  other  two  errors  cited  provide  examples  of  the  remaining 
principal  types  of  errors  encountered.  For  instance,  of  the  ten  errors 
pertaining  to  the  nature  of  the  chief  complaint,  three  were  due  to  the 
more  usual  type  of  "error  of  omission,"  since  they  reflect  the  absence 
of  a required  positive  response.  The  remaining  seven  of  these  ten  were 
errors  of  commission;  i.e.,  in  this  instance,  the  appearance  of  a chief 
complaint  that  was  not"sore  throat."  (The  chief  complaints  in  these 
instances  were,  however,  URI  complaints,  e.g.,  "cold,"  "head  cold,"  "flu.") 


*Within  the  context  of  the  present  study,  not  all  positively  marked  DCS 
asterisked  entries  were  considered  to  be  iype  A errors.  Some  of  the  aster- 
isked entries  (Figure  1,  page  13)  are  also  marked  with  a Such  en- 

tries may  be  addressed  specifically  within  the  algorithm  for  some  diagnoses. 
All  asterisked  entry  errors  which  were,  in  fact,  directly  and  specifically 
addressed  within  the  algorithm  were  categorized  as  Type  B,  logic-related 
errors  rather  than  Type  A errors. 
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As  regards  the  DCS  entry  "sore  throat  for  less  than  12  days,"  six  of  the 
eight  errors  encountered  were  also  errors  of  commission  due  to  the  exis- 
tence of  the  complaint  for  a period  longer  than  12  days  prior  to  the  time 
treatment  was  sought.  The  remaining  two  were  errors  of  omission  due  to 
the  lack  of  any  indication  as  to  the  duration  of  the  symptom. 

B. 3 Treatment-Related  Type  B Errors  for  Non-specific  Pharyngitis. 

B.3.1  In  Table  B-l,  the  preponderance  of  prescription-related  errors 
are  illustrated  by  the  findings  for  prescriptions  for  aspirin  and  aceta- 
minophen as  part  of  the  treatment  for  non-specific  pharyngitis.  Therein, 
the  findings  depicted  indicate  that  for  44.2  percent  of  the  patients, 
neither  aspirin  nor  its  alternate,  acetaminophen,  were  prescribed  for  15.4 
percent  of  the  patients,  acetaminophen  was  prescribed  instead  of  aspirin 
without  the  existence  of  any  entry  on  the  DCS  to  indicate  that  the 
medication-of-choice,  aspirin,  was  contraindicated.  In  the  writer's 
opinion,  however,  this  latter  type  of  error,  the  "unauthorized"  use  of 
the  alternate  medication,  is  less  serious  than  the  former  since  the  patient 
receives  at  least  some  medication  of  the  generic  type  indicated.  Nonethe- 
less, from  another  perspective,  that  of  costs  associated  with  the  program, 
this  practice  is  viewed  as  an  unquestionably  undesirable  one. 

B.3.2  The  information  appearing  in  Table  B-l  regarding  lack  of  pre- 
scriptions for  Cepacol  Lozenges  for  42.3  percent  of  the  patients  examples 
another  fairly  common  practice;  i.e.,  the  substitution  of  another  medica- 
tion not  on  the  DCS  for  one  which  already  appears  therein.*  As  regards 
this  particular  example,  it  was  often  the  case  that  Chloroseptic  spray 
(which  is  not  on  the  DCS  or  in  the  algorithm)  was  apparently  prescribed 
in  lieu  of  (and  at  other  times,  in  conjunction  with)  the  Cepacol  lozenges. 
(Chloroseptic  spray  was  prescribed  for  fully  50  percent  of  the  patients 
with  non-specific  pharyngitis.)  While  the  outcome  or  duration  of  the 
patient’s  illness  is  not  likely  to  be  differentially  affected  by  the  choice 
made  between  Cepacol  lozenges  or  Chloroseptic  spray,  or  the  choice  made 
between  aspirin  and  acetaminophen,  there  does  exist  at  least  one  reason  why 
the  former  is  the  medication-of-choice:  its  cost.  Therefore,  while  there 
may  be  little  medical  reason  for  choosing  one  or  the  other  of  these  two 
medications,  it  is  likely  that  non-adherence  to  the  algorithm's  guidance 
is  not  cost  effective.  Comparable  information  is  provided  for  the  re- 
mainder of  the  URI  diagnoses  evaluated  in  Tables  B-2  through  B-10. 


*Aside  from  the  apparent  non-DCS  "substitutions,"  there  were  also  17 
additional  non-DCS  prescriptions  encountered  among  the  52  cases  addressed 
in  Table  7.  They  were  as  follows:  Benadril  (1),  Codeine  (1),  Darvocet  (1), 
Debrox  Otic  (1),  Percogesic  (1),  "Snap  Caps"  (1),  Sudafed  (3),  and 
Zyambenenyst  (8) . 
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Table  B-l 


1 

I 

P 

LOGIC  ERRORS  FOR  ACUTE  PHARYNGITIS,  NON-SPECIFIC 


REQUIREMENTS 

TOTAL  NUMBER 
OF  ERRORS 

PERCENT  OF 
ALL  CASES  * 

SYMPTOMS 

Absence  of  positive  throat  culture 

within  last  10  days 

7 

13.4 

Absence  of  past  history  of  Rheumatic  fever 

6 

11.5 

Chief  complaint  is  "sore  throat" 

10 

19.2 

Sore  throat  for  less  than  12  days 

8 

15.4 

Treatment 

Aspirin  prescribed 

23 

44.2 

Acetaminophen  prescribed 

8 

15.4 

Cepacol  Lozenges  prescribed 

22 

42.3 

Throat  culture  requested 

2 

3.8 

Disposition  to  home  or  duty 

4 

7.7 

* Number  of  cases  is  52.  Errors  are  defined  as  the  absence  of  required 
entries  (including  errors  of  both  commission  and  omission). 


Table  B-2 


LOGIC  ERRORS  FOR  ACUTE  OTITIS  MEDIA 


REQUIREMENT 

TOTAL  NUMBER 
OF  ERRORS 

PERCENT  OF 
ALL  CASES  * 

Symptoms/Conditions 

Vertigo  absent 

5 

31.3 

Decreased  hearing  present 

7 

43.8 

Earache  for  less  than  seven  days 

7 

43.8 

Temperature  less  than  102°F 

14 

87.5 

Non-visibility  of  tympanic  membrane 

3 

18.8 

Tympanic  membrane  loss  of  landmarks 

10 

62.5 

Tympanic  membrane  redness 

1 

Treatment 


Instructions  to  return  to  clinic 

11 

68.8 

Aspirin  prescribed 

13 

81.3 

Actifed  prescribed 

8 

50.0 

^imetapp  prescribed 

1 

6.3 

Throat  culture  requested 

1 

6.3 

Ear  culture  requested 

1 

6.3 

* 


Number  of  cases  is  16.  Errors  are  defined  as  the  absence  of  required 
entries  (including  errors  of  commission  or  omission). 
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Table  B-3 


LOGIC  ERRORS  FOR  ACUTE  SEROUS  OTITIS  MEDIA 


REQUIREMENT 

TOTAL  NUMBER 

OF  ERRORS 

PERCENT  OF 
ALL  ERRORS  * 

Symptoms /Conditions 

Vertigo  absent 

6 

42.9 

Temperature  less  than 

102°F 

13 

92.9 

Tympanic  membrane  not 

visible 

4 

28.6 

Treatment 

Instructions  to  return 

to  clinic 

2 

14.3 

Actifed  prescribed 

14 

100.0 

Disposition  to  home  or 

duty 

1 

7.1 

* Number  of  cases  is  14.  Errors  are  defined  as  the  absence  of  required 
entries  (including  errors  of  both  commission  and  omission). 
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... 


Treatment 


toms /Conditions 


Vertigo  absent 
Temperature  less  than  102°F 
Ear  drainage  white-yellow 
Fluid  level  not  observed  behind 
tympanic  membrane 
Ear  canal  tenderness 


Cortisporin  Otic  prescribed 
Aspirin  prescribed 
Acetaminophen  prescribed 
Ear  cultures  requested 


* Number  of  cases  is  8.  Errors  are  defined  as  the  absence  of  required 
entries  (including  errors  of  commission  and  omission) . 


Table  B-5 


LOGIC  ERRORS  FOR  ACUTE  SINUSITIS 


REQUIREMENTS 

TOTAL  NUMBER 
OF  ERRORS 

PERCENT  OF 
ALL  CASES  * 

Symptoms /Conditions 

Blood  pressure  normal 

1 

10.0 

One  or  both  of  the  following  exist: 

4 

40 . 0 

Purulent  nasal  drainage  ** 

Nasal  nucosa  swollen  and  red 

Treatment 

Penicillin  V prescribed 

3 

30.0 

Aspirin  prescribed 

4 

40.0 

Neosynephrine  prescribed 

6 

60. 0 

* NumDer  of  cases  is  10.  Errors  are  defined  as  the  absence  of  required 
entries  (including  errors  of  commission  and  omission)  . 

**  Either  subjectively  reported  or  objectively  observable. 
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Table  B-6 


LOGIC  ERRORS  FOR  ALLERGIC  RHINITIS 


TOTAL  NUMBER 

PERCENT  OF 

REQUIREMENTS 

OF  ERRORS 

ALL  CASES  * 

Symptoms /Conditions 

Nasal  congestion  present 

2 

12.5 

Absence  of  purulent  nasal  drainage 

7 

A3. 8 

Nasal  nucosa  swollen  and  red 

5 

31.3 

Absence  of  sinus  tenderness 

Subjective  or  objective  evidence  of 

2 

12.5 

watery  nasal  drainage 

6 

37.5 

At  least  one  of  the  following: 

3 

18.8 

(a)  Recurrent/seasonal  nasal  suggestion 

(b)  Itching  eyes/nose 

(c)  Tearing  of  eyes 

Treatment 

Chlorpheniramine  Maleate  prescribed 

10 

62.5 

Disposition  to  home  or  duty 

2 

12.5 

* Number  of  cases  is  16.  Errors  are  defined  as  the  absence  of  required 
entries  (including  errors  of  commission  and  omission). 
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Table 


B-7 


LOGIC  ERRORS  FOR  NON-SPECIFIC  RHINITIS 


REQUIREMENTS 


TOTAL  NUMBER  PERCENT  OF 
OF  ERRORS  ALL  CASES  * 


Symptoms /Conditions 


Runny  nose  for  less  than  12  days  4 

Temperature  less  than  102°F  4 

One  or  more  of  the  following:  **  1 

(a)  Recurrent /seasonal  nasal  drainage 

(b)  Itching  of  eyes/nose 

(c)  Tearing  of  eyes 

Absence  of  the  following  in  combination:  **  1 

(a)  Nasal  mucosa  swollen  and  red 


(b)  Subjective  or  objective  evidence  of 
watery  nasal  drainage 

Treatment 

Actifed  prescribed  3 

Dimetapp  prescribed  3 

Disposition  to  home  or  duty  3 


44.4 

44.4 

11.1 


11.1 


33.3 

33.3 

33.3 


* Number  of  cases  is  9.  Errors  are  defined  as  the  absence  of  required 
entries  (including  errors  of  commission  and  omission) . 

**  Two  logic  paths  could  be  utilized  to  arrive  at  this  diagnosis  in 

accordance  with  the  AMOSIST  Manual.  Both  of  the  cited  multicriteria 
items  required  that  nasal  congestion  exist. 
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Table  B-8 


LOGIC  ERRORS  FOR  ACUTE  PHARYNGITIS, 
POSSIBLY  STREPTOCOCCAL 


TOTAL  NUMBER  PERCENT  OF 
REQUIREMENT  OF  ERRORS ALL  CASES  * 

Symptoms/Conditions 


Absence  of  positive  throat  culture 


within  last  10  days 

8 

A7.0 

Sore  throat  for  less  than  5 days 

2 

11.8 

Temperature  not  between  100°F  and  102°F 

11 

64 . 7 

Lymph  nodes  swollen 

2 

11.8 

Absence  of  posterior  lymph  node  swelling 

15 

88.2 

Tender  anterior  lymph  node 

1 

5.9 

Treatment 

Aspirin  prescribed 

10 

58.8 

Acetominophen  prescribed 

2 

11.8 

Cepacol  Lozenges  prescribed 

10 

58.8 

* Number  of  cases  is  17.  Errors  are  defined  as  the  absence  of  required 
entries  (including  errors  of  commission  or  omission). 
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Table  B-9 


LOGIC  ERRORS  FOR  BRONCHITIS 


REQUIREMENT 

TOTAL  NUMBER 
OF  ERRORS 

PERCENT  OF 
ALL  CASES  * 

Symptoms/ Conditions 

Cough  less  than  12  day  duration 

2 

10.0 

Cough  more 

than  12  days  and  improving 

7 

35.0 

Absence  of 

purulent,  bloody  sputum 

13 

65.0 

Absence  of 

chest  pain 

4 

20.0 

Absence  of 

unequal  breath  sounds 

5 

25.0 

Absence  of 

rales,  ronchi,  or  wheezing 

5 

25.0 

Treatment 

Robitussin 

DM  prescribed 

9 

45.0 

Dimetapp  prescribed 

1 

5.0 

Disposition  to  home  or  duty 

1 

5.0 

Number  of  cases  is  20.  Errors  are  defined  as  the  absence  of  required 
entries  (including  errors  of  commission  or  omission). 
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Table  B-10 


■ 


LOGIC  ERRORS  FOR  HEADACHE 


> REQUIREMENTS 


TOTAL  NUMBER  PERCENT  OF 
OF  ERRORS  ALL  CASES  * 


Headache  for  less  than  2 days 
Treatment 

Aspirin  prescribed 
Acetaminophen  prescribed 


5 71. A 


5 71.4 

1 14.3 


Number  of  cases  is  7.  Errors  are  defined  as  the  absence  of  required 
entries  (including  errors  of  both  commission  and  omission).  Additionally, 
since  all  of  the  above  data  were  from  data  collection  sheets  evidencing 
but  one  diagnosis  (and  no  physician  consultation),  all  7 of  them  are 
incorrect  since  the  logic  requires  that  at  least  one  of  eight  additional 
diagnoses  be  checked. 


APPENDIX  C 

TYPE  B,  LOGIC-RELATED  ERRORS  FOR  CYSTITIS 


C.l  General.  As  an  example  of  a non-URI  illness,  the  findings  per- 
taining to  the  only  other  single  diagnosis  occurring  with  any  substantial 
frequency,  cystitis,  are  presented  (Table  C-l).  An  additional  reason  for 
presenting  this  finding  is  the  appearance  of  therapeutic  drugs  (sulfisoxa- 
zole  and  ampicillin)  in  the  treatment  plan. 

C.2  Symptom-Related  Errors.  Two  of  the  symptoms  appearing  in  the 
table  were  of  the  type  requiring  a negative  response.  As  regards  the 
first  and  less  prevalent  error,  the  absence  of  vaginal  discharge,  no 
response  was  rendered  in  one  of  the  five  cases  while  positive  responses 
occurred  in  the  remaining  four.  Occurring  more  frequently  were  errors 
regarding  the  requirement  that  there  not  exist  two  or  more  previous  bladder 
infections.  Therein,  non-responses  accounted  for  2 of  the  11  errors,  while 
positive  responses  accounted  for  9.  Of  the  two  symptoms/conditions  re- 
quiring positive  responses,  the  failure  to  evidence  criterion-level  UA 
findings  was  the  most  frequently  encountered  error.  All  25  of  these  in- 
stances showed  negative  entries  in  the  relevant  spaces.  The  remaining 
positive-response-required  item,  "painful  urination  for  less  than  5 days" 
was  found  to  be  an  error  in  20  (52.6%)  of  the  DCSs.  In  the  majority  of  the 
instances,  (11  of  the  20)  they  were  checked  as  being  negative.  Non- 
responses occurred  five  times  and  for  the  remaining  four  cases,  the  length 
of  time  indicated  exceeded  the  five  dav  time  limit. 


C. 3 Treatment-Related  Type  B Errors. 

C.3.1  Among  the  treatment-related  requirements  for  initial  visits 
diagnosed  as  cystitis  is  the  need  to  request  a urinalysis  and  culture  to 
be  done  that  same  day.  As  shown  in  Table  C-l,  a positive  response  was  not 
indicated  for  this  DCS  entry  for  27  of  the  38  cases  (71.1%).  Among  these 
27,  two  showed  a negative  response,  25  evidenced  no  response  at  all,  and 
one  reflected  an  ambiguous  entry.  However,  among  these  27  it  is  also 
noted  that  nine  made  some  form  of  entry  or  written  annotation  in  the  upper 
portion  of  the  DCS  (near  the  UA  findings  DCS  entries)  that  suggest  that 
a UA  had  been  performed.  It  is  pointed  out,  however,  that  if  the  27 
erroneous  DCS  entries  are  reduced  by  this  amount  (the  eight  for  which 
some  UA  data  had,  apparently,  been  obtained),  the  remaining  19  comprise 
a very  substantial  50  percent  of  all  cases.  Hence,  even  if  the  data  are 
adjusted,  it  appears  that  one-half  of  the  patients  diagnosed  as  having 
cystitis  did  not  obtain  a urinalysis. 

C.3.2  The  treatment  plan  cited  in  the  AMOSIST  Manual  also  indicates 
that  the  patient  is  to  return  7 days  after  the  completion  of  treatment 
(medication  for  10  days)  and  obtain  a second  urinalysis.  At  the  time  of 
departure  from  the  clinic,  the  patient  is  to  receive  a laboratory  slip 
for  his  second  UA.  As  shown  in  Table  C-l,  these  aspects  of  the  treatment 
plan  were  largely  ignored.  The  DCS  entries  indicated  that  only  one  of 
the  38  patients  received  the  correct  return  instructions  (albeit  an 
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Table 


C-l 


LOGIC  ERRORS  FOR  CYSTITIS 


REQUIREMENTS 

TOTAL  NUMBER 
OF  ERRORS 

PERCENT  OF 
ALL  CASES  * 

Symptoms/ Conditions 

Absence  of  vaginal  discharge 

5 

12.5 

Painful  urination  for  less  than  5 days 

20 

50.0 

Absence  of  two  or  more  previous 
bladder  infections 

12 

40.0 

Obtained  or  requested  urinalyses  (UA) 
with  culture 

9 

22.5 

Either  (a)  UA  with  more  than  20  WBC/hpf 
or  (b)  US  with  8-20  WBC/hpf  and 
with  many  bacteria 

26 

65.0 

Treatment 

Ampicillin  prescribed 

15 

37.5 

Sulfisox  prescribed 

22 

55.0 

* 


Number  of  cases  is  38.  Errors  are  defined  as  the  absence  of  required 
entries  (including  errors  of  commission  or  omission). 
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additional  eight  received  incorrect  return  instructions)  and  only  about 
one-third  of  the  patients  received  the  laboratory  slip  required  to  ob- 
tain the  follow-up  UA. 

C.3.3  The  medications  prescribed  by  AMOSISTs  for  the  treatment  of 
cystitis  are  shown  in  Table  C-2.  As  evidenced  in  the  top  row,  no  patient 
was  treated  in  strict  accordance  with  the  algorithm;  i.e.,  given  a pre- 
scription for  sulf isoxazole  (or  ampicillin)  alone.  The  majority  of  the 
patients  (35  of  38)  received  pyridium  either  as  the  sole  medication  (17 
patients)  or  in  combination  with  either  sulf isoxazole  (8  patients), 

Septra  (1  patient)  or  ampicillin  (7  patients).  In  consonance  with  pre- 
i viously  cited  findings,  the  alternate  drug,  ampicillin,  was  prescribed 

without  apparent  documented  cause  in  lieu  of  the  drug-of-choice,  sulfi- 
soxazole,  in  six  of  the  seven  cases  where  it  was  utilized.  While  it  is 
not  apparent  from  the  table,  of  the  13  patients  for  whom  the  urinalysis 
(UA)  data  met  one  of  the  criteria  for  a diagnosis  of  cystitis,  three  did 
not  receive  either  sulfisoxazole,  ampicillin,  or  Septra.  Conversely,  these 
drugs  were  utilized  in  five  instances  wherein  the  UA  data  did  not  meet  the 
criteria.  The  findings  which  arouse  more  questions,  however,  are  those 
portrayed  in  the  bottom  two  lines  of  the  table.  These  address  prescrip- 
tions (or,  in  one  case,  the  lack  of  any  prescribed  medication)  for  non- 
therapeutic,  symptomatic  relief.  The  questionable  portion  of  the  data 
derives  from  the  observations  that  (a)  only  seven  of  the  20  patients  re- 
ceiving these  medications  were  indicated  to  have  had  a urinalysis  requested, 
and  (b)  17  of  the  20  were  reported  to  have  had  no  UA  findings  which  were 
indicative  of  the  presence  of  cystitis.  These  findings  were,  nonetheless, 
evidenced  upon  DCSs  which  showed  cystitis  as  the  diagnosis — a diagnosis 
which  is  predicated  upon  the  existence  of  specified  pathognomonic  UA 
findings.  The  data  are  obviously  inconsistent.  One  possible  explanation 
for  the  lack  of  pathognomonic  UA  findings  in  these  instances  is  that  the 
UAs  were,  in  fact,  neither  requested  nor  performed.  Albeit  this  absence 
would  be  consistent  with  the  lack  of  any  cystitis-positive,  UA  findings, 
such  would  be  contrary  to  the  logic  of  the  algorithm  since  the  algorithm 
requires  the  results  of  a UA  to  be  available  prior  to  making  a diagnosis 
of  cystitis.  An  alternative  explanation  might  be  that  the  UAs  were  per- 
formed and  the  results,  with  the  exceptions  of  two  cases,  were  all 
negative,  i.e.,  for  only  three  cases  did  the  UA  measurements  meet  the 
cystitis-related  criteria  employed  in  the  algorithm.  For  such  to  have 
been  the  case,  however,  most  of  the  "UA  with  culture  today"  entries  by 
the  AMOSISTs  would  have  to  be  incorrect,  since  17  of  these  20  indicated 
that  no  UAs  had  been  requested.  Obviously,  neither  posited  "explanation" 
must  have  been  true  for  all  cases;  each  may  have  applied  for  some  portion 
of  the  cases  in  question.  Nevertheless,  regardless  of  which  circumstance 
(if  any)  applied,  it  remains  contrary  to  the  logic  of  the  algorithm  for 
the  diagnosis  "cystitis"  to  have  been  rendered  in  all  but  three  of  the 
20  cases  appearing  in  the  last  three  lines  of  Table  C-2. 

C.3.4  The  last  two  columns  of  Table  C-2  pertain  to  the  post- 
treatment UA  and  culture  called  for  in  the  treatment  of  cystitis.  Appar- 
ently 12  of  the  15  patients  receiving  therapeutic  drug  treatment  received 
the  necessary  urine  culture  and  sensitivity  laboratory  slip  required  to 
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Table  C-2 


MEDICATIONS  PRESCRIBED  FOR  CYSTITIS 


DRUGS  PRESCRIBED  (NO.  PTS) 

URINALYSIS 

REQUESTED 

UA  CRITERIA 
MET 

RETURN 

INSTRUCTED 

FOLLOW-UP 
URINALYSIS  a 

Sulfisoxazole  (0) 

- 

- 

- 

- 

Sulflsoxazole  + Pyridium 
(3) 

8 

5 

1 

7 

Septra  + Pyridium  (1) 

1 

1 

0 

1 

Ampicillin  + Pryidium  C (7) 

6 

4 

d 

0 

4 

Pyridium  e (19) 

5 

2 

0 f 

0 

Urispas  ^ (3) 

2 

1 

h 

0 

1 

TOTAL:  (38) 

23 

13 

1 

13 

a.  Urine  culture  requested  for  17  days  after  initiation  of  treatment. 


b.  Two  other  DCSs  provided  patient  return  instructions,  but  both  were 
inaccurate,  i.e.,  one  was  for  PRN  and  the  other  was  for  14  days 
instead  of  17  days. 

c.  One  of  these  seven  also  included  a prescription  for  Parafon  Forte. 

Only  one  evidenced  a contraindication  for  the  "drug-of-choice" , 
Surfisoxazole. 

d Included  one  instruction  for  PRN  and  one  instruction  without  specific 
number  of  days  cited. 

e.  One  patient  also  received  Darvocef;  another  received  a second  drug 
which  was  indecipherable  on  the  "Xerox"  copy.  One  also  received  TCN. 

f.  Two  included  PRN  instructions. 

g.  Two  of  these  indicated  that  they  were  MD  prescribed,  but  no  physician's 
signature  or  initials  appeared  on  the  DCS. 

h.  One  instruction  was  PRN. 
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obtain  the  follow-up,  post-treatment  tests.  The  DCSs  for  only  one  of 
these  patients,  however,  evidenced  entries  which  indicated  that  they  had 
been  appropriately  instructed  to  return.  For  those  receiving  only  sympto- 
matic medication,  the  findings  are  equally  poor.  None,  apparently,  received 
appropriate  instructions  regarding  their  return,  and  only  one  was  provided 
the  necessary  laboratory  slips  for  follow-up  urinalysis.  (It  is  noted, 
however,  that  one  could  rather  obtusely  employ  a "devil's  advocate" 
position  and  question  the  validity  of  these  "errors"  by  asking  if,  in  fact, 
it  is  necessary  to  either  (a)  therapeutically  treat  patients  who  did  not 
evidence  the  required  UA  findings,  or  (b)  instruct  them  to  return  for  fur- 
ther UAs  17  days  later.  While  this  point  may  be  well  taken,  the  question 
remains  why  they  were  diagnosed  as  having  cystitis  in  the  first  place.) 
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APPENDIX  D 


AMIC  RETROSPECTIVE  AUDIT 
FINDINGS  FOR  THE  URI  ALGORITHM 


Table  D-2  cont 


I 


KUK3ER  OF 

PHYSI-  FREQUENCY  OF  NOS-DIRECTED  FREQUENCY  OF  UTILIZATION 


Table  D-2  cont 


PHYSI- 

FREQUENCY  OF 

NUMBER  OF 
NON-DIRECTED 

FREQUENCY  OF  UTILIZATION 

DIAGNOSIS 

CIAN 

CONSULT 

OCCURRENCE 
Site  1 Site  2 Total 

RETURN  VISITS 
Site  1 Site  2 Total 

Site  1 

SF  600 
Site  2 

Total 

OLD  DCSs 
Site  1 Site  2 

Total 

11/0 

Yes 

1 

- 

1 

l - 1 

- 

- 

- 

- 

- 

No 

- 

- 

- 

- 

- 

- 

- 

- 

- 

11/16 

Yes 

- 

- 

- 

- 

- 

- 

- 

- 

- 

No 

- 

1 

1 

_ 

- 

- 

- 

- 

- 

12/13 

Yes 

1 

5 

6 

- 

- 

2 

2 

-• 

- 

No 

1 

7 

8 

2 2 

- 

- 

- 

- 

- 

12/14 

Yea 

’ 

- 

- 

- 

- 

- 

- 

- 

No 

- 

1 

1 

- 

- 

- 

- 

- 

- 

12/16 

Yes 

- 

2 

2 

1 1 

- 

- 

- 

- 

- 

No 

- 

- 

- 

- 

- 

- 

- 

- 

- 

12/0 

Yes 

- 

- ■ 

a*  mm 

- 

- 

- 

- 

- 

- 

No 

1 

- 

1 

1 - 1 

- 

- 

- 

- 

- 

13/14 

Yes 

- 

1 

1 

- 

- 

- 

- 

- 

- 

No 

1 

3 

4 

1 - l 

- 

- 

- 

- 

- 

13/15 

Yes 

- 

1 

1 

- 

- 

- 

- 

- 

- 

No 

- 

- 

- 

_ 

- 

- 

- 

- 

- 

13/16 

Yea 

1 

1 

2 

- 

- 

- 

- 

- 

- 

No 

2 

- 

2 

- 

- 

- 

- 

- 

- 

13/0 

Yes 

2 

1 

3 

1 1 

- 

1 

1 

- 

- 

No 

1 

- 

• 1 

- 

- 

- 

- 

- 

- 

14/0 

Yes 

- 

- 

- 

- 

- 

- 

- 

- 

- 

No 

2 

- 

2 

1 - 1 

- 

- 

- 

- 

- 

15/16 

Yes 

- 

- 

_ 

- 

- 

- 

- 

- 

No 

1 

- 

1 

- 

- 

- 

- 

- 

- 

15/0 

Yes 

- 

1 

1 

- 

- 

- 

- 

- 

- 

No 

1 

- 

1 

- 

• - 

- 

- 

- 

- 

16/0 

Yes 

- 

1 

1 

- 

- 

- 

- 

- 

- • 

No 

1 

- 

1 

. 

- 

- 

- 

- 

- 

All 

Yes 

18 

36 

54 

2 5 7 

0 

6 

6 

0 1 

1 

TOTALS: 

No 

31 

17 

48 

102 

7 2 9 

7? 

0 

0 

0 

~6 

0 0 

0 

T 

69 
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Table  D-3 


I 


AMIC  RETROSPECTIVE  AUDIT  FINDINGS  FOR  DCSa  HAVING  MORE  THAN  TWO  DIAGNOSES  FROM  THE  RESPIRATORY  ALGORITHM 


L 
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APPENDIX  E 

EVALUATORY  COMMENTS  REGARDING  THE  OVERLAY 
TRANSPARENCY  APPROACH  TO  THE  AUDITING  OF 
TREATMENT  DATA  COLLECTION  SHEETS 

E.l  General . The  following  paragraphs  present  a brief  critique  of 
the  overlay  transparency  method  of  auditing  treatment  DCSs.  Discussed 
are  the  issues  of  DCS  sample  constraints,  time  requirements,  and  overlay 
construction. 

E. 2 DCS  Selection  and  Availability. 

E.2.1  One  of  the  principal  drawbacks  of  the  proffered  overlay  approach 
to  a large-scale  audit  of  AMOSIST-completed  DCSs  is  the  limited  number  of 
DCSs  to  which  it  might  be  applicable.  The  initial  constraint  is  that  au- 
dits are  applicable  only  to  DCSs  completed  without  the  need  for  physician 
consultation.  As  depicted  in  Table  E-l  it  is  clear  that  nearly  two-thirds 
of  all  DCSs  are  apt  to  be  excluded  from  the  logic-related  audit  on  these 
grounds  alone.  If  the  audit  is  performed  solely  upon  DCSs  evidencing  but  a 
single  diagnosis,  the  number  available  for  audit  purposes  is  decreased  to 
but  one-fourth  of  all  AMOSIST-completed  DCSs.  While  it  may  appear  that  the 
sample  of  auditable  DCSs  could  be  significantly  increased  by  relaxing  the 
"single  diagnosis"  constraint  employed  in  the  present  study  and  include 
two-diagnosis  DCSs  as  well,  the  present  data  suggest  that  such  is  not  the 
case.  The  criterion  employed  in  the  present  study  was  that  at  least  five 
DCSs  had  to  exist  to  justify  the  effort  entailed  in  developing  an  overlay 
transparency.  Were  the  same  criterion  to  be  utilized  in  the  two-diagnosis 
situation,  it  is  readily  apparent  that  only  the  URI  category  of  illness 
could  be  addressed  since  there  does  not  even  exist  a total  of  five  two- 
diagnosis  DCSs  in  any  of  the  other  categories,  let  alone  a unique  combi- 
nation of  two  for  which  five  existed.  Unfortunately,  the  subsample  of  48 
two-diagnois  URI  DCSs  (Table  E-l)  provided  only  two  combinations  (Acute 
Pharynsitis  in  conjunction  with  Acute  Serous  Otitis  Media  and  Acute  Phar- 
yngitis in  conjunction  with  non-specific  Rhinitis)  which  occurred  five  or 
more  times.  In  sum,  were  the  sample  of  AMOSIST-only  completed  DCSs  expan- 
ded to  include  two-diagnosis  DCSs,  the  number  of  evaluable  DCSs  would  have 
been  increased  by  less  than  five  percent. 

E.2.2  While  the  above  discussion  began  with  an  assertion  that  the 
small  number  of  records  available  constituted  a drawback  to  the  "overlay" 
approach,  the  use  of  the  term  "drawback"  to  describe  the  circumstance  was 
rhetorical  since  it  is  in  actuality  a limiting  condition  (rather  than  an 
adverse  one)  which  applies  not  only  to  the  manual  overlay  approach  but 
also  to  any  DCS  logic  auditing  procedure  employed  using  the  present  DCSs. 

As  pointed  out  previously,  a deficit  inherent  in  the  present  system  of 
data  collection  is  that  there  is  no  way  to  determine  whether  or  not  an 
AMOSIST's  consultation  with  a physician  resulted  in  a decision  to  overrule 
the  logic  of  the  algorithm.  Thus,  for  any  auditing  process  used  with  the 
present  DCSs  (be  it  manual  or  computerized)  only  about  one-fourth  of  all  DCSs 
would  be  aduitable.  (The  single-diagnosis  constraint  remains,  however,  a limita- 
tion of  an  overlay  auditing  approach  using  the  present  type  of  DCS.  As 
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Table  E-l 


DISTRIBUTION  OF  RETROSPECTIVE  AUDIT  SAMPLE:  AMOSIST-TREATED  CASES  BY  ALGORITHM, 
NUMBER  OF  DIAGNOSES  PER  DCS,  AND  PilYSICIAN  CONSULTATION  REQUIREMENTS 


TOTAL 

SINGLE 

TWO 

THREE 

OR  MORE 

PHYSICIAN 

NO. 

DIAGNOSES 

DIAGNOSES 

DIAGNOSES 

ALGORITHM  CONSULTED 

PTS 

NO. 

Z OF  TOTAL 

NO. 

Z OF  TOTAL 

NO.  % 

OF  TOTAL 

Eye  (1) 

YES 

14 

12 

85.8 

2 

14.2 

- 

- 

NO 

7 

5 

71.4 

2 

28.6 

- 

- 

Respiratory  (2) 

YES 

234 

105 

44.9 

54 

23.1 

75 

32.0 

NO 

216 

111 

51.4 

48 

22.2 

57 

26.4 

Gastrointestinal 

YES 

44 

43 

97.7 

1 

2.3 

(3) 

NO 

20 

20 

100.0 

- 

- 

- 

Genitourinary 

YES 

69 

56 

81.2 

13 

18.8 

- 

- 

(4) 

NO 

35 

33 

94.4 

2 

5.6 

- 

Musculo-Skeletal , 

YES 

6 

6 

100.0 

_ 

_ 

_ 

_ 

Spine  (5) 

NO 

1 

1 

100.0 

- 

- 

- 

Musculo-Skeletal , 

YES 

63 

63 

100.0 

_ 

— 

- 

— 

Extremity  (6) 

NO 

21 

21 

100.0 

- 

- 

- 

Skin-Regional 

YES 

73 

69 

94.5 

4 

5.5 

_ 

- 

(7) 

NO 

15 

14 

93.3 

1 

6.7 

- 

Skin-General 

YES 

56 

54 

96.4 

2 

3.6 

_ 

_ 

(8) 

NO 

31 

31 

100.0 

- 

- 

- 

- 

All  (1-8) 

YES 

559 

408 

73.1 

76 

13.5 

75 

13.4 

NO 

346 

236 

68.2 

53 

15.3 

57 

16.5 

TOTALS 

905 

644 

71.1 

129 

14.3 

132 

14.6 
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will  be  discussed  later,  however,  this  limitation  might  be  avoided  if  the 
format  of  the  DCSs  were  changed.) 

E.3  Time  Considerations.  Another  consideration  related  to  the  use 
of  an  overlay  auditing  process  is  the  amount  of  time  necessary  to  retain 
the  medical  records  for  the  purpose  of  evaluating  them  or  extracting  the 
required  information  from  them.  It  is  likely  that  an  overlay  approach 
would  permit  a daily  audit  to  be  made  of  100  percent  of  all  single  diag- 
nosis, AMOSIST-completed  DCSs  (DCSs  which  reflect  no  physician  consulta- 
tion) at  a cost  of  approximately  one  man-hour  per  day  with  no  need  to 
retain  the  medical  records  within  the  AMIC  for  any  amount  of  time  beyond 
the  end  of  normal  workday  once  tfie  auditor  becomes  familiar  with  the 
use  of  the  overlays  and  properly  schedules  his  time.  (The  one  man-hour 
estimation  is  based  upon  a workload  of  100  patients  per  day  from  which, 
as  described  earlier,  only  25  would  be  legitimate  candidates  for  logic- 
related  audit.) 

E.4  Overlay  Transparency  Construction.  One  difficulty  encountered 
in  the  present  study  that  should  be  addressed  if  a decision  were  to  be 
made  to  develop  and  export  an  overlay  evaluation  system  is  that  of  the 
site-to-site  distortions  in  the  DCSs,  due  apparently,  to  differences 
among  the  various  copiers  and  printers  employed  to  effect  local  repro- 
duction of  the  DCS  originals  first  supplied  by  AHS.  It  is  noted,  however, 
that  centrally  distributed  DCSs  would  eliminate  the  problem  of  distortion- 
induced  overlay  mismatch.  Albeit  budget  allocations  might  have  to  be  re- 
distributed to  permit  AHS  to  effect  the  centralized  printing  and  distribution 
of  all  forms  required,  it  may  be  that  quantity  discounts  from  vendors  would 
permit  this  function  to  be  performed  at  little  overall  increase  in  cost  to 
the  command.* 


i 


*(Presently,  each  hospital  must  locally  reproduce  all  DCSs.)  Were  all 
overlay  transparencies  to  be  centrally  produced  and  distributed  from 
AHS,  the  recipients  would  have  to  suffer  the  continuing  minor  irritation 
of  having  to  adjust  the  overlays  two,  three,  or  more  times  for  each  DCS 
evaluated  in  order  to  keep  them  properly  aligned. 
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